io-oidis-hub

Backend server for the Oidis synchronization hub. Provides the authentication system, data access layer, agent reverse tunneling, webhooks, email engine, GraphQL API, and system monitoring infrastructure.

Package Purpose

io-oidis-hub is the central backend that solution projects extend. It provides:

• Authentication & Authorization — User management, groups, organizations, JWT tokens, OAuth2, 2FA, SSO
• HTTP Server & API — REST API with Bearer token auth, GraphQL API with per-operation authorization
• Agent Tunneling — VPN-like WebSocket reverse proxy for devices behind firewalls
• Email Engine — Template-based async email queue (registration, password reset, invitations, reports)
• Webhooks — Adapter-based webhook processing with hub-to-hub synchronization
• Data Layer — 17 MongoDB models with GraphQL schema auto-generation, PostgreSQL alternative
• External Connectors — Slack, MS Teams, logging, reports, certificates, file transfer
• System Monitoring — Database-backed audit logs, crash reporting (Sentry), environment state tracking
• Print Pipeline — PDF generation via Puppeteer

Dependency Position

io-oidis-commons ← io-oidis-localhost ← io-oidis-hub ← solution backends (e.g. io-oidis-{project})
                                              ↑
                                        io-oidis-hub-gui (frontend, noServer: true)

Hub extends io-oidis-localhost (the core HTTP server runtime) and is the topmost backend layer. Solution backends subclass Hub's Loader, AuthManager, and HttpServer to add project-specific logic.

Source Structure

source/typescript/Io/Oidis/Hub/
├── AuthProcessor/       — Authentication partials (Users, Groups, Organizations, OAuth2, 2FA, Tokens)
├── Connectors/          — External service integrations (Slack, MS Teams, Certs, Files, Reports)
├── DAO/
│   ├── Converters/      — Model-to-reference converters (AuthToken, FileEntry, Group)
│   ├── GraphQL/         — Dynamic GraphQL schema builder
│   ├── Models/          — 17 MongoDB models (User, Group, Organization, FileEntry, etc.)
│   ├── Mongoose/        — MongoDB driver (MongoContext)
│   ├── Postgres/        — PostgreSQL driver (PostgresContext)
│   ├── Pages/           — Base page data access
│   └── MailEngineDAO.ts — Email template data access
├── Enums/               — LoaderEventType
├── HttpProcessor/
│   ├── Resolvers/       — 16 HTTP resolvers (REST, GraphQL, Forwarding, FileUpload, etc.)
│   ├── ResponseApi/     — GraphQL response formatter
│   └── HttpServer.ts    — Hub HTTP server with security filtering
├── Interfaces/          — IProject configuration, loader events, mail engine config
├── LogProcessor/        — SystemLog (DB audit), CrashAdapter (Sentry)
├── Primitives/          — Decorators (AgentsRegister, @WebhookAdapter, @GQLMethod)
├── RuntimeTests/        — Self-validation tests (Agents, Auth, Config, Selfupdate)
├── Structures/          — ProgramArgs
├── Utils/               — AuthManager, EnvironmentHelper, MailEngine, Threads, Puppeteer, etc.
├── Webhooks/            — WebhooksManager, adapter pattern, OidisHook adapter
└── Loader.ts            — Main entry point

Loader & Initialization Lifecycle

The Loader class extends the localhost Loader and orchestrates the full application startup. Its progress is tracked by EnvironmentStatePool (Utils/EnvironmentStatePool.ts) as an ordered state chain:

IDLE → STARTING → DB_CONNECTING → DB_CONNECTED → DB_PREPARED → AGENTS_INIT → [ADMIN_INIT → ADMIN_ENABLED] → LOADED
failure states: DB_NOT_CONNECTED, DB_CONNECTION_ERROR

Environment State Chain

The last step of the chain is the end state — the environment is loaded (IsEnvironmentLoaded(), HTTP 200 from FrontEndLoader instead of 503, OnComplete to loading listeners) only once that step is reached. LOADED is the end state of the Hub itself, but an application can append its own steps after it, so never compare the current state with LOADED directly — use IsComplete() / getEndState(). To ask whether an intermediate step has already happened (for example whether the database has ever connected), use HasReached(EnvironmentState.DB_CONNECTED) — it is false before the step, in a failure state and for a state outside the chain.

Rules applied by State($value):

• Transitions may skip steps forward (for example AGENTS_INIT → LOADED when no admin init is needed).
• A backward transition or a state that is not part of the chain is ignored with a warning.
• A failure state notifies listeners through OnError with the step label as the message; the chain may continue forward from it.
• A listener registered after the chain completed or failed receives OnComplete / OnError immediately, so the init page never waits for a transition that already happened.

Each step is an IEnvironmentStep (io-oidis-services, Connectors/EnvironmentConnector.ts): state, optional label (progress text on the init page), route (the init page switches to it, e.g. ADMIN_INIT → /AdminInit) and failure. The front end reads the chain through EnvironmentConnector.getStateChain().

Applications shape the chain in their Loader constructor through getStatePool(), or by overriding defineChain() in their own pool returned from initStatePool():

constructor() {
    super();
    this.getStatePool().Append(
        {label: "Loading lexicons.", state: EnvironmentState.LEXICONS_INIT},
        {state: EnvironmentState.LEXICONS_LOADED});
}

protected async onDbPrepared() : Promise<void> {
    await super.onDbPrepared();                 // Hub steps up to LOADED
    this.getStatePool().State(EnvironmentState.LEXICONS_INIT);
    await this.getLexiconHelper().getLanguages(true);
    this.getStatePool().Complete();             // LEXICONS_LOADED - the end state
}

Available chain operations: Append(), InsertAfter(), InsertBefore(), Replace(), Remove(). Anchors must exist and states must be unique, otherwise the call throws. EndState($value) is deprecated — it truncates the chain after an existing state or appends an unknown one.

Puppeteer (PDF print) browser installation runs in the background after the chain reached LOADED/ADMIN_INIT, so a Chrome download on the first start does not delay the init page.

Boot Sequence

1. Constructor — Creates EventsManager, InMemoryAdapter (browser) or PushNotifications (server), initializes EnvironmentStatePool
2. processProgramArgs() — Handles agent mode, registers OnDbConnected and OnDbPrepared event handlers, opens database connection
3. onDbConnected() — Initializes GraphQL schema (RegisterModels()), runs DB upgrade if version mismatch, syncs system groups, creates admin token. On worker threads: starts activation check and cleanup cron jobs
4. onDbPrepared() — Creates cron worker thread (if authManagerWorker.enabled), cleans old system logs, syncs configurations and app settings, initializes agent service account, initializes Puppeteer (if enabled), starts MailEngine and Webhooks processing loops
5. onEnvironmentLoad() — Adds log adapters: ForwardingAdapter (browser) or CrashAdapter (server, unless noReport: true). Registers RuntimeHandler symbol mapping

Lazy-Initialized Singletons

The Loader uses a self-replacing getter pattern for lazy initialization:

public getAuthManager() : AuthManager {
    if (!ObjectValidator.IsSet(this.authManager)) {
        this.authManager = this.initAuthManager();
        this.getAuthManager = () : AuthManager => {
            return this.authManager;
        };
    }
    return this.authManager;
}

This pattern is used for AuthManager, WebhooksManager, MailEngine, EnvironmentHelper, GraphQL, and AdminInitHelper.

Events System

The Loader exposes four lifecycle events via getEvents():

Event	Fires When	Handler receives	Typical Use
OnDbConnected	Database connection established	—	Initialize data-dependent services
OnDbPrepared	System groups synced, admin token created	—	Start background processing
OnNewAgentConnected	Agent registers via WebSocket	the agent (id, name, packageInfo, …)	Act on a specific agent coming online
OnAgentDisconnected	Agent WebSocket closes	the agent id (string)	Clean up agent state

Loader.getInstance().getEvents().OnDbPrepared(async () : Promise<void> => {
    // your initialization code
});

Subscribe from onDbPrepared()

Override onDbPrepared() in the solution Loader and register the handlers there — it runs once the DB is ready, so a handler may already touch models. This is also where solutions start their crons.

OnNewAgentConnected fires only for a genuinely new agent registration (a second instance of an already-registered agent is logged and skipped), and the handler receives the agent itself — so a solution can react to one specific peer instead of polling getAgentsList():

protected async onDbPrepared() : Promise<void> {
    await super.onDbPrepared();
    this.getEvents().OnNewAgentConnected(async ($agent : any) : Promise<void> => {
        if (String($agent?.name || "") !== "MyAgent") {
            return;
        }
        try {
            await new MyCollector().ForceSync();
        } catch (ex) {
            // Never let a handler failure escape into the registration path — the agent must finish
            // registering even when this first pull fails.
            LogIt.Warning("sync on agent connect failed: " + ((<any>ex)?.message || ex));
        }
    });
}

Use it to pick an agent up the moment it returns (reboot, self-update, network blip) instead of waiting for the next poll interval.

Extending the Hub

Solution projects subclass key Hub classes to add project-specific behavior:

Loader

Override init*() factory methods to provide custom implementations:

export class Loader extends HubLoader {

    protected initAuthManager() : AuthManager {
        return AuthManager.getInstance(); // your extended AuthManager
    }

    protected initMailEngine() : MailEngine {
        return new MailEngine(); // your extended MailEngine
    }

    protected initWebhooksManager() : WebhooksManager {
        return WebhooksManager.getSingleton(); // your extended WebhooksManager
    }

    protected async onDbUpgrade($oldVersion : string) : Promise<boolean> {
        // your DB migration logic
        return await super.onDbUpgrade($oldVersion);
    }
}

AuthManager

See Authentication for detailed guidance on extending the auth system with project-specific permissions.

HttpServer

Override isRequestAllowed() to add custom request filtering or resolveRequest() to add custom routing logic.

---

See also: Authentication | Request Forwarding | HTTP Processing Pipeline | Localhost Overview | Hub Overview