Skip to main content

Email System (MailEngine)

Hub includes a template-based asynchronous email engine that handles user registration, password reset, invitations, account lifecycle notifications, feedback, and crash reports.

Architecture​

MailEngine (orchestrator)
  ├── SendMail          — SMTP transport (nodemailer)
  ├── MailEngineDAO     — Template data access (JSONP-based templates)
  ├── StringReplace     — Variable substitution in templates
  ├── EmailRecord       — DB queue (MongoDB model)
  └── MJML              — HTML email rendering

The engine operates in two modes:

  • Queued — Most email types are saved as EmailRecord documents and processed by the async loop
  • Direct — Critical emails (e.g. REPORT_FAILURE) are sent immediately, bypassing the queue

Email Types​

TypeTriggerDescription
REGISTERUser registrationAccount activation link
RESET_PASSWORDPassword reset requestPassword recovery link
RESET_PASSWORD_CONFIRMPassword successfully resetConfirmation email
INVITATIONGroup invitation sentInvitation link with group and owner info
ACTIVATIONAccount nearing expiryWarning with days remaining and re-activation link
DEACTIVATIONAccount deactivatedNotification with re-activation link
REMOVEDAccount deletedNotification with sign-up link
RESTOREDAccount restoredNotification with sign-in link
REPORT_FAILUREUnhandled exceptionCrash report with stack traces (sent directly to admins)
FEEDBACKUser submits feedbackFeedback message forwarded to admin emails
PUBLIC_FEEDBACKPublic feedback submissionSame as feedback but from public context
NEW_AUTH_CODENew auth code generatedOne-time code for 2FA
AUTH_CODE_RESENDAuth code resend requestedRe-sent code with optional CC

Public API​

const mail : MailEngine = Loader.getInstance().getMailEngine();

// User lifecycle
await mail.Register($email, $activationToken);
await mail.ResetPassword($email, $resetToken);
await mail.ResetPasswordConfirm($email);
await mail.Invitation($email, $groupName, $ownerName, $inviteToken);
await mail.Activation($email, $token, $daysLeft);
await mail.Deactivation($email, $token);
await mail.Removed($email);
await mail.Restored($email);

// System
await mail.ReportFailure($logTraces);
await mail.Feedback($email, FeedbackType.General, $message, $uuid);

// Auth codes
await mail.SendNewCode($email, $dataId, $code);
await mail.ResendNewCode($email, $dataId, $code, $ccEmail);

Template System​

Email templates are stored as JSONP configuration files managed by MailEngineDAO. Each email type has:

FieldPurpose
subjectEmail subject line (supports variables)
titleDisplay title in email header
headerHTML header content
htmlBodyMain body content — raw HTML or MJML, see Body Content Contracts
plainBodyPlain text fallback
templatePath to MJML template file (optional, falls back to default)
navigateTextCTA button text
contentModeauto (default) or raw to opt out of body normalization

Variables are substituted using StringReplace with StringReplaceType.VARIABLES syntax (<? @var variableName ?>). Available variables include:

  • domain — Application domain URL
  • link — Action URL as clickable <a> tag
  • accountsAddress — Support email from config
  • email — Recipient email
  • url — Raw action URL
  • Custom per-type: group, owner, days, message, code, category, uuid

Templates are rendered through MJML (responsive email framework) to produce cross-client compatible HTML.

Body Content Contracts​

htmlBody is injected into the template at the body.content variable. Where a template places that variable decides what the body must contain — the two forms are not interchangeable:

body.content sits inhtmlBody must beExample
<mj-text>raw HTMLActivate your account:<br><? @var link ?>
<mj-section>, <mj-body>, <mj-wrapper>MJML<mj-column><mj-text>Activate…</mj-text></mj-column>

Mixing them is not a cosmetic problem. Raw HTML placed among MJML children fails strict MJML validation; the engine then falls back to soft validation, which discards the offending nodes, and the mail is delivered with a heading and an empty body.

Automatic Normalization​

MailTemplateNormalizer resolves the slot from the template and shapes the body to match, so a configuration written for one contract still renders under the other:

  • HTML body + MJML slot → wrapped in <mj-column><mj-text>…</mj-text></mj-column> (plus <mj-section> for an mj-body slot)
  • MJML body + mj-text slot → unwrapped to raw HTML; an mj-button becomes an <a> link
  • body already matching the slot → passed through unchanged

Because a matching body is never touched, normalization is a no-op for any configuration that is already correct. Every reshape is reported through LogIt.Warning — treat it as a defect to fix in the configuration, not as a supported state.

Set contentMode: "raw" on a content entry (or on the configuration root) to disable normalization for hand-tuned bodies.

Inherited Templates​

A configuration using extendsConfig deep-merges the parent's contents, so a mail type it does not override inherits the parent's body and the parent's per-type template path. An inherited template path would render the mail with the parent application's branding, so MailEngineDAO.getTemplatePathFor() ignores it and falls back to the application-level template. A per-type template declared by the application itself always wins.

This is why an application extending the Hub configuration does not need to re-declare contents.feedback.template to keep its own branding.

When a mail type is sent without a target URL (feedback and similar), the mj-button block is removed from the template before rendering, so no dead CTA is produced. Self-closing mj-button defaults declared in mj-attributes are preserved.

Async Processing Loop​

The MailEngine runs an async processing loop that picks up queued EmailRecord documents:

mail.StartLoop();     // Start with configured fetchTick
mail.StartLoop(5000); // Start with custom interval (ms)
mail.StopLoop();      // Stop processing

Loop Lifecycle​

  1. StartLoop() — Checks sendmail.enabled and sendmail.fetchTick > 0
  2. First tick fires after 1 second (fast initial processing)
  3. Subsequent ticks use configured fetchTick interval
  4. Each tick: query all active EmailRecord documents → render template → send via SMTP → remove record on success
  5. StopLoop() — Clears the loop timer

Loop Failure Suppression​

A tick that fails as a whole (typically EmailRecord.Find during a DB outage) reports one ERROR on the healthy→failing transition and a WARNING with the failed-tick count on recovery; the ticks in between log at INFO only, so a minutes-long outage produces two Sentry events instead of one per tick. The same pattern guards the webhooks loop and SystemLog persistence (which additionally echoes the original trace message at its intended level when it cannot be stored, so the root cause is never swallowed). A SystemLog write that fails before the database connected for the first time (for example a trace written while an uptime check hits the booting server) logs at INFO only and is not counted as an outage — a failed or timed-out initial connection is reported by the database driver itself.

Automatic Retry and Dead-Letter​

A failed send does not dead-letter the record immediately. The loop books the attempt on the record (Attempts, LastError with the SMTP/template root cause, LastAttemptAt) and schedules the next automatic retry via NextAttemptAt using the retryBackoffSec schedule (the last value repeats), so transient failures such as network outages heal without admin intervention. Only after maxSendAttempts failed attempts is the record dead-lettered (Active = false) with its root cause preserved for the dashboard's Reason column.

Additional rules:

  • Records waiting inside their backoff window are skipped without consuming a batchSize slot.
  • The age dead-letter (maxQueueAgeHours) keys on max(Created, LastAttemptAt), so a retrying or manually re-activated record cannot expire untried; expired records carry an "Expired: ..." reason.
  • The dashboard's per-row Activate action resets the retry budget (Attempts, LastError, NextAttemptAt) and stamps LastAttemptAt, making manual retry meaningful even for old records.

Loop States​

StateMeaning
StoppedLoop is not running
RunningCurrently processing records
IdleWaiting for next tick

Recipient Routing​

  • User emails (register, reset, invite, etc.) → sent to the specific user's email
  • Admin emails (feedback, crash reports) → sent to all addresses in sendmail.admins list
  • Blacklist — Addresses in sendmail.blackList are silently excluded from the to: recipients (not from copies, see below)

Recipient groups and duplicate reduction​

getMailList() returns string[][] — an array of groups, and each group is dispatched as one separate e-mail with the whole group in its to: header. Two built-in reduction mechanisms apply:

  • reducesMailList($list, $ignore) strips the primary recipient from a copy group (case-insensitive substring match), so the main addressee never receives the same mail twice via the copy send.
  • Duplicate addresses within one group are tolerated — the mail transport (SMTP envelope handling) collapses identical recipients of a single message, so an override merging several sources (e.g. sendmail.admins plus organization owners) does not need to deduplicate exhaustively.

The blacklist filter runs per group at send time, after getMailList() resolves.

Copy recipients (CC)​

Put copyTo (a string or string[]) into the $environment passed to sendMail(). directSend() copies it into the transport options and SendMail joins an array with ", " into the cc: header of every group mail. For queued types the whole environment is stored as JSON in EmailRecord.Environment and parsed back when the loop sends it, so copyTo survives the queue.

// inside a MailEngine subclass; REPORT_READY is a solution-defined EmailType
const env : any = {name: $reportName};
if ($copyTo.length > 0) {
    env.copyTo = $copyTo;
}
return await this.sendMail($email, $url, EmailType.REPORT_READY, env);
warning

copyTo is passed through unfiltered: sendmail.blackList is applied only to the to: recipients. Drop blacklisted and duplicate addresses (including the primary recipient) yourself before setting copyTo. Copies also receive the same body, so tracking pixels in the template fire for CC readers too.

Extending​

Solution projects can extend MailEngine to:

  • Add custom email types via new EmailType values
  • Override allowedEmailTypes() to filter which types are active
  • Override directEmailSend() to control which types bypass the queue
  • Override getMailList() to customize recipient routing per email type
  • Override getTemplate() or getContentEnvironment() to inject custom template variables
  • Override normalizeContent() to control how bodies are shaped for the template slot
  • Call setTemplatePath() to use project-specific template configurations

Configuration​

See the sendmail section in Hub Configuration for SMTP settings, fetch tick, domain, admins list, and blacklist.

Contact Form Intake (public REST)​

ContactFormConnector exposes a public two-phase intake used by the static Oidis webs (oidis-solutions, geodron, ...) to deliver contact-form messages through the MailEngine feedback pipeline instead of mailto: or third-party services:

GET  /api/contact-form/token     -> {token, expiresIn}
POST /api/contact-form/message   -> {ok: true} | {error: <code>}

Flow and protections:

  1. The web fetches a token on page load. Tokens are HMAC-signed (SHA-256, random per-boot secret), carry their issue timestamp and expire after tokenTtlSeconds (default 900).
  2. The message POST must present the token. It is rejected while younger than minSubmitSeconds (default 3 — bots submit instantly; the page-load fetch anchors the age) and each token is single-use.
  3. Per-client sliding-window rate limits (maxPerHour, default 5; maxPerDay, default 20) keyed by X-Forwarded-For.
  4. Origin allow-list: browsers send Origin; only exact matches from contactForm.origins receive Access-Control-Allow-Origin. Clients POST with Content-Type: text/plain carrying JSON so the request stays CORS-simple (no OPTIONS preflight).
  5. Payload {token, email, message, name?, company?, phone?, site?, source?, lang?} is validated and length-capped, then delivered via MailEngine.Feedback() (EmailType.FEEDBACK) to sendmail.admins.

The whole surface is off unless the contactForm configuration section enables it:

contactForm: {
    enabled: true,
    origins: ["https://www.oidis.io", "https://geodron.cz"],
    tokenTtlSeconds: 900,
    minSubmitSeconds: 3,
    maxPerHour: 5,
    maxPerDay: 20
}

Anonymous REST dispatch follows the same model as POST /api/login — the request itself authenticates through the service-account fallback of the Hub RESTResolver, while the route handler is public.

Local Development (without Docker)​

In the Docker DEV environment, the sendmail config points to the smtp container on port 25. When developing locally without Docker, you need a standalone SMTP server to test email sending.

Nodemailer App is a free desktop SMTP server for development. It captures all outgoing emails in a local inbox — nothing is actually delivered.

  1. Download the latest release from github.com/nodemailer/nodemailer-app/releases
  2. Launch the application — it starts an SMTP server on localhost:1025 by default
  3. Configure the sendmail section in your local config to connect to it:
sendmail: {
    serviceType: "",
    user: "",
    pass: "",
    secure: false,
    defaultSender: "Oidis HUB-DEV<noreply@oidis.io>",
    location: "localhost",
    port: 1025
}

The key differences from the Docker DEV config are location changing from "smtp" (Docker service name) to "localhost", and port changing from 25 to 1025.

A few more things decide whether an email really lands in the Nodemailer inbox:

  • Project credentials. When you create a project in Nodemailer App, it shows its own SMTP port, user and password — put those into port, user and pass instead of the defaults above.
  • domain. Links in emails (invitations, password reset) are built from sendmail.domain. Set it to the local address of your application (e.g. "http://localhost:9315"), otherwise the links point to the server configured in the project config.
  • Sending is switched on in the administration, not in the config. With target.preferDBSettings: true (the Hub default) EnvironmentHelper.SyncAppSettings overwrites sendmail.enabled on every start with the value stored in the database (AppSettings.Sendmail.Enabled, false by default). Turn it on in Admin → App settings → Mail Engine → Save settings, or with Enable sending in the Email section. The value is stored in the database, applies immediately and survives restarts; Reload from server config switches it back to the config value.
  • An email sent while sending is off is dropped, not queued. MailEngine.sendMail only logs "Email send skipped…", so switch sending on before you invite a user.

See also: Hub Configuration | Hub Database Models | Hub Backend Overview