irdb/doc/architecture.md

Architecture

Audience: operators running IRDB, and engineers building replacement frontends. Read this first if you're new to the codebase.

System overview

IRDB is an IP reputation database — it ingests abuse reports about specific IP addresses, applies a decaying weighted score per category, and distributes tailored block lists to firewalls and proxies. Reporters (web servers, IDS, fail2ban-style agents) push events; each consumer (firewall, proxy) pulls a block list shaped by a named policy that decides which categories and thresholds count.

The system is shipped as a small Docker Compose stack. Reporters and consumers talk JSON to the api over an authenticated REST surface; human operators use a thin PHP UI that calls the same api with a service token plus an impersonation header. The split is deliberate: the UI is replaceable, the api is the contract.

Container topology

flowchart LR
    subgraph clients[Clients]
        rep[Reporters<br/>web/IDS/fail2ban]
        con[Consumers<br/>firewalls/proxies]
        adm[Admins<br/>browser]
    end

    subgraph stack[Compose stack]
        ui[ui<br/>PHP+Twig BFF<br/>:8080]
        api[api<br/>Slim+FrankenPHP<br/>:8081]
        migrate[migrate<br/>one-shot Phinx]
        sched[scheduler<br/>busybox crond]
        mysql[(mysql<br/>optional)]
    end

    db[(SQLite or MySQL)]

    rep -- POST /api/v1/report ----> api
    con -- GET /api/v1/blocklist --> api
    adm -- HTTPS --> ui
    ui -- service token + X-Acting-User-Id --> api
    sched -- POST /internal/jobs/tick --> api
    migrate --> db
    api --> db
    mysql -.-> db

Five services in compose; only two run continuously (api + ui). migrate is one-shot (restart: "no"); scheduler is opt-in via the compose.scheduler.yml overlay; mysql is opt-in via uncommented config. Single-host SQLite deployments run with just migrate + api

• ui; the data volume is shared between migrate and api.

Where state lives

Where	What
irdb-data Docker volume	SQLite database file /data/irdb.sqlite, GeoIP MMDBs at /data/geoip/. Owned exclusively by the api container; migrate mounts it briefly to apply schema changes.
mysql-data Docker volume	Optional MySQL /var/lib/mysql. Replaces the SQLite path when DB_DRIVER=mysql.
ui container's writable layer	PHP file-backed sessions (/tmp per the FrankenPHP base). Tied to a single replica; sticky sessions required if scaling UI horizontally.
.env	All secrets (service token, internal job token, OIDC client secret, MySQL password, local-admin Argon2id hash).
Browser localStorage	Light/dark theme preference only. Session is in a Secure cookie.

Notably absent: the ui container has zero persistent state. Replace its image and nothing of operational importance is lost. The api owns everything.

Stable surfaces vs replaceable parts

The contract for future frontends, integrators, and replacement UIs. Anything in the "stable" column should not break across v1 minor releases.

Surface	Stability	Notes
/api/v1/* paths and request/response shapes	stable	Additive changes only within v1. Breaking changes ship as v2. OpenAPI is canonical.
Token kinds (reporter, consumer, admin, service)	stable	Strings are persisted in api_tokens.kind and used in audit logs.
RBAC roles (viewer, operator, admin)	stable	Persisted in users.role and api_tokens.role.
users shape and oidc_role_mappings semantics	stable	Future SPAs/native UIs upsert via /api/v1/auth/users/upsert-oidc the same way the BFF does today.
Service-token + X-Acting-User-Id impersonation	stable	The pattern any BFF replacement uses; the API doesn't trust the header without a service token.
Error envelope {error, details?}	stable	Validation errors include details; auth failures don't.
Audit actor_kind enum	stable	user, admin-token, reporter, consumer, system. Used by external SIEM exporters in future work.
Twig template names under ui/resources/views/	replaceable	Tied to the current UI implementation; replacement UIs render however they like.
UI route paths under /app/*	replaceable	Browser-only; not part of any contract.
Internal class names, method signatures, file layout	replaceable	Refactoring is fair game.
/internal/jobs/*	replaceable	Scheduler-only; no public guarantee. Not in OpenAPI.
Twig globals, htmx attributes, Alpine components	replaceable	Implementation details of this UI.

Why this split

Backend-for-Frontend (BFF) pattern. The api is a pure JSON service: no HTML, no sessions, no cookies. The ui is a thin browser-facing process that owns OIDC handshakes, browser sessions, CSRF, and Twig rendering — but no business logic, no scoring, no database. Every admin page on the UI fetches from the api with the service token plus the acting user's id.

Why this matters:

1. The UI is replaceable. A team that wants Vue, Svelte, or a Tauri desktop app can replace the ui container without touching the api. The contract is the OpenAPI document and the impersonation pattern. See frontend-development.md.

2. No HTML in the api. Other clients (firewalls, fail2ban, monitoring) call the same admin endpoints the UI does. If the api started rendering Twig, those integrations would suddenly carry HTML they don't want.

3. Auditing is honest. The api records the impersonated user as the actor (actor_kind=user) when called via the BFF, never the service token. An admin token used directly records as actor_kind=admin-token. Future external authentication paths can plug in alongside without breaking the audit trail.

4. The api can scale independently. With MySQL, the api is stateless and can run as N replicas behind a load balancer; the job_locks table mediates between them. The UI is typically single-replica because of file-backed sessions.

The trade-off: every UI request makes one or more outbound calls to the api, which costs latency and an HTTP hop. In practice the containers run on the same Docker network and the per-request cost is sub-millisecond.

Disaster Recovery

The api owns one stateful resource: the database (SQLite file on a Docker volume, or MySQL). Everything else — sessions, tokens, GeoIP caches — is recoverable without a backup.

What carries irreplaceable state

Resource	Where	Recovery
Reports + scores	reports, ip_scores tables	DB backup
Manual blocks + allowlist	manual_blocks, allowlist	DB backup
Policies + thresholds	policies, policy_category_thresholds	DB backup
Categories	categories	DB backup
Reporters / consumers / users	reporters, consumers, users	DB backup
API tokens (hashes only)	api_tokens	DB backup; raw values gone — re-issue
Audit log	audit_log	DB backup
Service token (UI_SERVICE_TOKEN)	.env file	Back up .env or regenerate
OIDC client secret	.env file	Re-fetch from Entra app registration
Local admin password hash	.env file	Regenerate via password_hash
MaxMind/IPinfo credentials	.env file	Re-fetch from provider
GeoIP MMDBs	irdb-data volume	refresh-geoip job re-downloads
Browser sessions	UI container's writable layer	Discarded on restart — users re-login

Recovery checklist

1. Provision the host: install Docker; restore .env (or generate a fresh one — see README.md → Quick start and rotate the service token).
2. Restore the database per the relevant block in README.md → Backups. For SQLite, drop the restored file into the irdb-data volume; for MySQL, pipe the dump through mysql.
3. Bring up the stack: docker compose up -d. The migrate container runs Phinx idempotently — safe to run on a restored DB that's already at HEAD; it'll exit immediately with "no migrations to run".
4. Verify by calling /healthz on both containers and signing in to the UI. Trigger refresh-geoip from Settings (or accept the first scheduled run) so GeoIP enrichment is populated again.
5. Re-issue tokens for any reporter or consumer that lost its raw token (the database has the SHA-256 hash, but the original string isn't recoverable). Consumers will keep failing 401 until they're given new tokens.

What we DON'T support

• Encryption at rest of the SQLite file. The volume's host-level disk encryption is the right layer; SPEC §15.
• Audit log signing / tamper-evidence. The audit_log is append-only at the application layer but a sufficiently privileged attacker with DB access can rewrite history. Future work.
• Cross-region replication. Docker volumes are local; high availability requires a managed MySQL with replication plus an api topology that can take advantage of it.

Where the rest of the docs live

• api-overview.md — the public API surface, with worked examples.
• auth-flows.md — every authentication flow (machine, BFF, OIDC, local admin), Entra setup, future user-token sketch.
• frontend-development.md — the headline doc for replacement UIs.
• api-reference.md — short. Pointer to OpenAPI plus the things OpenAPI doesn't cleanly express (rate limits, ETag semantics, impersonation header, response conventions).

The OpenAPI document itself is at api/public/openapi.yaml and served at /api/v1/openapi.yaml with a viewer at /api/docs.