# IRDB — IP Reputation Database

Self-hosted service that ingests abuse reports from many sources (web servers,
IDS, fail2ban-like agents) and distributes tailored, decay-weighted block lists
to firewalls and proxies. Ships as a Docker Compose stack — `api` (JSON
backend), `ui` (PHP+Twig BFF), and optional `mysql` / `scheduler` sidecars.

For who: ops engineers who run their own infra, want a single place to
collect abuse signal across hosts, and need consumer-shaped output (one
firewall = one tailored list).

The full design is in [`SPEC.md`](./SPEC.md). Per-milestone progress is in
[`PROGRESS.md`](./PROGRESS.md). Documentation for operators and future
frontend authors lives in [`doc/`](./doc/).

---

## Quickstart (≈5 minutes)

```bash
git clone <this-repo> irdb && cd irdb
cp .env.example .env

# Generate secrets — see "Generating secrets" below for the exact commands.
$EDITOR .env

docker compose -f docker-compose.yml -f compose.scheduler.yml up -d
```

That's it. The UI is at `http://localhost:8080`, the api at
`http://localhost:8081`, and the API reference viewer at
`http://localhost:8081/api/docs`.

Log in with the local admin credentials you set in `.env`
(`LOCAL_ADMIN_USERNAME` / `LOCAL_ADMIN_PASSWORD_HASH`). OIDC works too —
see [`doc/auth-flows.md`](./doc/auth-flows.md).

---

## Generating secrets

Every value in `.env` marked with a comment "32-byte hex" or similar is a
secret you need to generate. Use these one-liners:

```bash
# 32-byte hex (UI_SECRET, APP_SECRET, INTERNAL_JOB_TOKEN)
openssl rand -hex 32

# IRDB-format service token (UI_SERVICE_TOKEN — looks like irdb_svc_…)
docker compose run --rm -T api php -r 'require "/app/vendor/autoload.php";
    echo (new App\Domain\Auth\TokenIssuer())->issue(App\Domain\Auth\TokenKind::Service);'

# Local admin password hash (LOCAL_ADMIN_PASSWORD_HASH — Argon2id)
php -r "echo password_hash('your-admin-password', PASSWORD_ARGON2ID);"
# Note: in your .env, double every $ in the hash to $$ so docker-compose
# variable substitution doesn't eat it.
```

The api validates required env vars on boot; misconfiguration crashes
`docker compose up` rather than the first user click.

---

## First-time setup — reporter and consumer

Once the stack is up, log in to the UI as the local admin and:

1. **Create a category** if the seeded ones don't fit. `Categories →
   New`. Slugs are kebab-ish (`brute_force`, `web_attack`).
2. **Create a reporter** at `Reporters → New`. Trust weight defaults to
   `1.0`; lower it to dampen a noisy source.
3. **Create a token for the reporter**: `Tokens → New`, kind = reporter,
   pick the reporter you just made. **Copy the raw token now** — it is
   shown once and never displayed again.

Then post a report from the command line:

```bash
curl -X POST http://localhost:8081/api/v1/report \
  -H "Authorization: Bearer irdb_rep_…" \
  -H "Content-Type: application/json" \
  -d '{"ip":"203.0.113.42","category":"brute_force","metadata":{"url":"/wp-login.php"}}'
# → 202 {"report_id":1,"ip":"203.0.113.42","received_at":"…"}
```

For the distribution side: create a consumer (`Consumers → New`, pick a
policy — `moderate` is a good default), create a consumer token, then
pull the blocklist:

```bash
curl http://localhost:8081/api/v1/blocklist -H "Authorization: Bearer irdb_con_…"
# → text/plain: one IP or CIDR per line
```

Add `?format=json` for richer per-entry data. Use the `ETag`
+ `If-None-Match` round-trip to skip retransfer if nothing changed.

End-to-end examples for fail2ban, iptables-restore, nginx, and HAProxy
are in [`examples/`](./examples/).

---

## Reverse proxy in production

The default compose deployment exposes plain HTTP on `:8080` (UI) and
`:8081` (api). For production, front them with Caddy / nginx / Traefik
and route by hostname:

```
reputation.example.com       → ui:8080
reputation-api.example.com   → api:8081
```

A working Caddy config is in
[`examples/reverse-proxy/Caddyfile`](./examples/reverse-proxy/Caddyfile)
— it terminates TLS via Let's Encrypt and forwards both hostnames.

Single-hostname routing (everything under `reputation.example.com` with
`/api/*` → api, `/*` → ui) is documented as an alternative in the
example file.

---

## MySQL (optional)

SQLite (default) is fine for single-host deployments. For networked
storage or multi-replica `api` scaling, switch to MySQL:

1. Uncomment the `mysql` service block in `docker-compose.yml`.
2. Set `DB_DRIVER=mysql` and the `DB_MYSQL_*` vars in `.env`.
3. `docker compose up -d`.

The `migrate` container runs the same Phinx migrations against MySQL on
boot. Phinx detects the adapter; the only schema-shape difference is
adapter-aware `DATETIME(6)` vs SQLite `TEXT` for timestamps (handled
in `BaseMigration`).

> **Networked storage warning**: SQLite's WAL mode is unreliable on
> NFS / SMB / EFS. If you use networked storage, use MySQL.

---

## OIDC (Microsoft Entra ID)

Walkthrough in [`doc/auth-flows.md`](./doc/auth-flows.md), sections
"Entra setup" and "OIDC configuration variables". Set
`OIDC_*` vars in `.env`, restart the `ui` container, and the login page
gains a "Sign in with Microsoft" button.

Group → role mapping lives in the `oidc_role_mappings` table. Until
the dedicated admin UI ships, populate it directly:

```bash
docker compose exec -T api sh -c \
  "sqlite3 /data/irdb.sqlite \\
    \"INSERT INTO oidc_role_mappings(group_id, role) VALUES('<entra-group-id>', 'admin');\""
```

Default role for unmapped users is `viewer`; set
`OIDC_DEFAULT_ROLE=none` in `.env` to deny logins instead.

---

## Scheduling

Periodic jobs (recompute scores, refresh GeoIP, prune audit log,
enrich pending IPs) are exposed at `/internal/jobs/*` on the api. Three
ways to drive them:

**Sidecar (default in compose.scheduler.yml)** — busybox `crond` posts
to the api once a minute. No host setup required. Started by:

```bash
docker compose -f docker-compose.yml -f compose.scheduler.yml up -d
```

**Host cron** — install
[`examples/scheduler/host.crontab`](./examples/scheduler/host.crontab)
into the system crontab. Suitable when you don't want a sidecar.

**systemd timer** — install
[`examples/scheduler/irdb-tick.service`](./examples/scheduler/irdb-tick.service)
and
[`examples/scheduler/irdb-tick.timer`](./examples/scheduler/irdb-tick.timer)
into `/etc/systemd/system`, then `systemctl enable --now irdb-tick.timer`.

All three drive the same `/internal/jobs/tick` endpoint, which is the
dispatcher: it asks `job_runs` what's due and invokes those jobs in
turn. The endpoint is bound to RFC1918 + loopback only (Caddyfile
config in `api/docker/Caddyfile`); external requests get `404`.

---

## Updating

Pull new code and rebuild — Docker's layer cache makes the rebuild fast
when only app code changes:

```bash
git pull
docker compose -f docker-compose.yml -f compose.scheduler.yml up --build -d
docker compose logs -f          # Ctrl-C once migrate exits 0 and api/ui are healthy
```

`up --build -d` rebuilds local images and recreates only the containers
whose image hash or config changed. The `migrate` container reruns Phinx
automatically; new migrations in `db/migrations/` are picked up in order.
The `irdb-data` volume persists, so SQLite state, GeoIP MMDBs, and the
audit log carry forward across updates.

Don't use `docker compose restart` to pick up new code — that just bounces
the existing containers. Don't use `docker compose down -v` either — that
deletes the volume.

Edge cases (failed migrations, force-rebuild, rollback, fixing volume
ownership after a uid change, disk cleanup, scheduler ops) are covered in
the [admin manual](./doc/admin-manual.md).

---

## Backups

The api's persistent state lives in one of two places.

**SQLite (default)** — online-safe via the SQLite backup API:

```bash
docker compose exec api sh -c \
  'sqlite3 /data/irdb.sqlite ".backup /data/irdb-backup.sqlite"'
docker compose cp api:/data/irdb-backup.sqlite ./irdb-backup-$(date +%F).sqlite
```

The `.backup` command is the only correct way to copy a live SQLite
database with WAL — it quiesces the journal and produces a consistent
snapshot.

**SQLite — whole-volume tarball** (alternative, requires the api to be
stopped or quiesced):

```bash
docker compose stop api
docker run --rm -v irdb-data:/data -v "$(pwd):/backup" alpine \
  tar czf /backup/irdb-backup.tar.gz -C /data .
docker compose start api
```

Restore: `docker compose down`, drop or empty the volume, then
extract:

```bash
docker run --rm -v irdb-data:/data -v "$(pwd):/backup" alpine \
  sh -c 'rm -rf /data/* && tar xzf /backup/irdb-backup.tar.gz -C /data'
docker compose up -d
```

**MySQL**:

```bash
docker compose exec mysql sh -c \
  'mysqldump --single-transaction --routines --quick \
    -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' \
  > irdb-mysql-$(date +%F).sql
```

Restore (api must be stopped during restore so it doesn't observe a
half-loaded schema):

```bash
docker compose stop api migrate
docker compose exec -T mysql sh -c \
  'mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD" "$MYSQL_DATABASE"' \
  < irdb-mysql-2026-04-29.sql
docker compose up -d migrate api
```

The schema is small (under 20 tables); a multi-GB dump is a red flag —
`audit_log` and `reports` are the only tables that grow with use, and
`cleanup-audit` + `score_hard_cutoff_days` bound them.

**What NOT to back up**:
- **Rotating tokens** — the `api_tokens` table is included in the
  database backup automatically, but the *raw* token strings shown
  once on creation aren't recoverable. If a token is lost, revoke and
  re-issue.
- **GeoIP DBs** (`/data/geoip/*.mmdb`) — re-downloadable via the
  `refresh-geoip` job on first run after restore.
- **`UI_SERVICE_TOKEN`** etc. live in `.env`, not in the database;
  back up the env file separately if you need to redeploy from a
  blank node.

See [`doc/architecture.md` → Disaster Recovery](./doc/architecture.md#disaster-recovery)
for the end-to-end recovery checklist.

---

## Architecture

Three containers (`api`, `ui`, `migrate`) plus optional `mysql` and
`scheduler` sidecars. The split is a **BFF pattern**: `api` is the
JSON backend (owns the database, business logic, RBAC); `ui` is the
browser-facing PHP+Twig frontend that holds sessions and forwards
calls with a service token + impersonation header.

Full diagram + rationale in [`doc/architecture.md`](./doc/architecture.md).

---

## API contract

The OpenAPI document is the source of truth: visit
**`http://localhost:8081/api/docs`** for the interactive viewer, or
fetch the YAML at `/api/v1/openapi.yaml`.

Higher-level prose (token kinds, auth flows, common conventions) lives
in [`doc/api-overview.md`](./doc/api-overview.md). For machine clients
specifically, [`examples/`](./examples/) has copy-paste shell + Python
scripts for both reporters and consumers.

---

## Replacing the UI

The PHP+Twig UI is **deliberately replaceable**. The api's contract,
auth model, and token kinds are stable; the UI is one of several
possible frontends (Vue, native desktop, mobile clients are
explicitly anticipated).

If you're building a replacement, start at
[`doc/frontend-development.md`](./doc/frontend-development.md). It
describes the three integration patterns (BFF replacement, SPA + thin
BFF, direct API), the minimum API surface a fully-featured UI uses,
and what NOT to do.

---

## Local CI

```bash
./scripts/ci.sh
```

Runs cs/stan/test for both subprojects and verifies the docker compose
images build. Requires Docker; no PHP/Node toolchain needed on the host.

```bash
./scripts/check-doc-endpoints.sh
```

Doc accuracy guard: greps `doc/*.md` for `/api/v1/*` paths, fails if any
mentioned path is not in `api/public/openapi.yaml`. Run after editing
either side.

```bash
./tests/e2e/demo.sh
```

End-to-end smoke check — boots compose, creates a reporter+consumer
+tokens, posts a report, pulls the blocklist, and tears down. Mirrors
the quickstart documented above.

---

## License

Licensed under the [Apache License, Version 2.0](./LICENSE).

Copyright 2026 Alessandro Chiapparini &lt;irdb@chiapparini.org&gt;.
See [`NOTICE`](./NOTICE) for required attribution when redistributing.