irdb/files/M13-polish-openapi-docs.md

M13 — Polish, OpenAPI, Documentation

Fresh Claude Code agent prompt. M12 must be complete and committed. Estimated effort: large. Documentation is a real deliverable; budget time for accuracy.

Mission

Generate openapi.yaml and serve it at /api/v1/openapi.yaml plus a viewer at /api/docs. Write the README with quickstart and operational guides. Write every doc/*.md file as specified in SPEC.md §16. Ship sample reporter scripts and firewall consumer configs in examples/. By the end, a fresh clone goes from git clone to a working blocklist via documented steps in under 10 minutes.

Before you start

1. Verify M12:

   git log --oneline -12
   cd api && composer test && cd ..
   cd ui  && composer test && cd ..
   

2. Read SPEC.md §16 in full. The required outline of each doc/*.md is the contract for this milestone. Outline-skipping is a hard fail.

3. Skim SPEC.md §3 (architecture, for the Mermaid diagram you'll embed in doc/architecture.md), §6 (API surface for OpenAPI), §8 (auth flows for doc/auth-flows.md).

Tasks

1. OpenAPI spec

Place at api/public/openapi.yaml. Generate, don't hand-write — but also don't pull in a heavy framework. Two acceptable approaches:

• Annotation-based: use zircote/swagger-php to generate from PHP attributes on the controllers. Add the dep, sprinkle attributes, run vendor/bin/openapi src -o public/openapi.yaml as a composer script. CI runs this and fails if the result doesn't match what's committed.
• Hand-curated: a single api/openapi.yaml.template.php file that produces the YAML from PHP code (a structured array you serialize). Simpler; less boilerplate.

Pick one and document the choice in PROGRESS.md.

Coverage:

• All Public endpoints (/api/v1/report, /api/v1/blocklist).
• All Admin endpoints (/api/v1/admin/*).
• All Auth endpoints with x-internal: true extension and a clear "UI BFF only" description.
• Do not document /internal/jobs/* — those are private. Mention in the spec description that they exist but are out of scope for the public contract.

Schemas: define all request/response shapes (Report, Blocklist, Token, Policy, User, AuditEntry, error envelopes). Reuse via $ref.

2. OpenAPI viewer

Add a tiny route in api/src/Application/Public/DocsController.php:

• GET /api/v1/openapi.yaml — serves the YAML file with Content-Type: application/yaml. Public; no auth.
• GET /api/docs — serves an HTML page that loads Stoplight Elements or RapiDoc from a CDN-vendored npm static asset (no external CDN). Both are single-script viewers.

Pick one (RapiDoc is smaller; Stoplight Elements is prettier). Document the choice.

3. README

Replace the M01 stub. New contents (in order):

1. One-paragraph elevator pitch — what IRDB is, who it's for.

2. Quickstart — the 5-minute path:

   git clone ...
   cp .env.example .env
   # edit .env: generate secrets, optionally configure OIDC
   docker compose -f docker-compose.yml -f compose.scheduler.yml up -d
   # browse to http://localhost:8080, log in
   

3. Generating secrets — the exact openssl rand and php password_hash commands.

4. First-time setup — create a reporter, get a token, send a report; create a consumer, get a token, fetch the blocklist.

5. Reverse proxy setup — point to examples/reverse-proxy/Caddyfile.

6. MySQL setup — uncomment the section in compose; set DB_DRIVER=mysql.

7. OIDC setup — point to doc/oidc.md.

8. Scheduling — host cron, systemd timer, sidecar overlay; point to examples/scheduler/.

9. Backups — what to back up: the irdb-data volume (or MySQL); how to restore.

10. Architecture — point to doc/architecture.md.

11. API contract — point to /api/docs viewer and doc/api-overview.md.

12. Replacing the UI — for future Vue/native/mobile, point to doc/frontend-development.md.

13. License — TBD (leave a placeholder).

4. doc/ files (the real work)

Write each file according to its required outline in SPEC.md §16. Quality bar applies: every snippet must run as-is against docker compose up; no TODOs; ≤500 lines per file.

• doc/architecture.md — system overview, container topology (Mermaid), where state lives, stable-vs-replaceable surfaces table, why-this-split rationale.
• doc/api-overview.md — base URL/versioning, auth summary, endpoint groups, common conventions (envelopes, pagination, ETag, rate limits, IP normalization), worked curl examples for: posting a report, pulling a blocklist (text + JSON), admin search via service-token impersonation, admin search via admin-kind token. Pointer to OpenAPI.
• doc/auth-flows.md — overview table, sequence diagrams (Mermaid) for: machine reporter, admin token, UI BFF (OIDC + local), Entra setup walkthrough (extract from M08's doc/oidc.md and merge), local admin guidance, future user-token flow sketch marked NOT IMPLEMENTED, CSRF/sessions/CORS notes.
• doc/frontend-development.md — the headline doc. Read-this-first; three integration patterns (BFF replacement, SPA + thin BFF, direct API + future user tokens) with worked pseudocode for the BFF replacement; minimum API surface checklist; CORS configuration; local dev (run only api, point a separate frontend dev server at it); migration path for swapping UIs at runtime; what NOT to do list (no business logic in frontend, no service token in browser, etc.).
• doc/api-reference.md — short. Pointer to OpenAPI as canonical; documents what OpenAPI doesn't cleanly express: rate-limit headers, ETag semantics, the X-Acting-User-Id impersonation header convention, response envelope conventions for current and future batched endpoints.

If doc/oidc.md was created in M08, delete it — its content goes into doc/auth-flows.md per SPEC.md §16.

5. Examples

In examples/:

• reporters/curl.sh — a copy-paste shell script: takes an IP and category as args, posts a report. Reads IRDB_URL and IRDB_TOKEN from env.
• reporters/python.py — same in Python. Single file, no deps beyond urllib. Example of a fail2ban-action wrapper inline as a comment.
• reporters/bash-fail2ban.sh — drop-in fail2ban action.
• consumers/iptables-restore.sh — pulls the blocklist, builds an ipset, atomic-replace via ipset restore.
• consumers/nginx-deny-include.sh — pulls and writes an include file with deny directives, reloads nginx.
• consumers/haproxy-acl.sh — pulls and updates an HAProxy ACL file.
• scheduler/host.crontab, scheduler/irdb-tick.service, scheduler/irdb-tick.timer — already stubbed in M01, fill in real content.
• reverse-proxy/Caddyfile — production-ready Caddy config fronting api and ui.

Each script has a header comment explaining usage. Each is shell-checked (run shellcheck) and tested at least manually.

6. End-to-end demo test

Add tests/e2e/demo.sh (and a CI job that runs it) that automates the README quickstart:

1. docker compose -f ... up -d
2. Generate admin token via CLI
3. Create reporter + consumer + tokens
4. Submit reports
5. Trigger recompute job
6. Pull blocklist; assert non-empty
7. docker compose down -v

7. Doc accuracy CI check

Add a CI job that:

• Greps doc/*.md for endpoint paths; compares against the OpenAPI document. Any path in docs not in the spec → fail. (The other direction is OK; not every endpoint needs prose.)
• Greps for token kind strings (reporter, consumer, admin, service); ensures spelling matches code.
• Optional: dead-link checker on the docs.

Implementation notes

• Mermaid in GitHub-rendered Markdown: use mermaid fenced code blocks. Test by rendering the file on a GitHub PR.
• Pseudocode in frontend-development.md: keep it language-neutral or use a minimal Node/Express snippet that's clearly a sketch, not a runnable thing. The point is the pattern, not a working app.
• Avoid screenshots: explicitly forbidden by SPEC §16 quality bar — they go stale.
• OpenAPI versioning: declare version: 1.0.0 in the spec; bump for breaking changes. Document the additive-only policy.
• Don't describe what doesn't exist: every claim in docs must match the as-built code. If you find a discrepancy mid-writing, fix the code, not the docs.

Out of scope (DO NOT)

• Marketing site / landing page. README is enough.
• Versioned docs site (Docusaurus, MkDocs). The Markdown files in doc/ are the docs.
• Auto-generating client SDKs. Future work.
• Tutorials beyond the quickstart in README. The doc files are reference, not tutorials.
• Adding new endpoints. If something feels needed for docs that isn't in the code, stop and reconsider.
• New dependencies beyond zircote/swagger-php (if you go that route). Record in PROGRESS.md.

Acceptance

cd api && composer cs && composer stan && composer test && cd ..
cd ui  && composer cs && composer stan && composer test && cd ..

# OpenAPI is valid
docker run --rm -v "$(pwd)/api/public:/spec" \
  redocly/cli:latest lint /spec/openapi.yaml

# Doc files exist and are non-empty
for f in architecture api-overview auth-flows frontend-development api-reference; do
  test -s "doc/$f.md" || { echo "missing or empty: doc/$f.md"; exit 1; }
done

# Each doc file is ≤500 lines
for f in doc/*.md; do
  L=$(wc -l < "$f"); [ "$L" -le 500 ] || { echo "$f too long: $L lines"; exit 1; }
done

# Doc accuracy: no stale endpoints
# (run the CI check script you wrote)
./scripts/check-doc-endpoints.sh

# E2E demo script
docker compose down -v
./tests/e2e/demo.sh
docker compose down -v

# /api/docs serves a viewer
docker compose up -d
sleep 15
curl -sf http://localhost:8081/api/v1/openapi.yaml | grep -q "openapi:"
curl -sf http://localhost:8081/api/docs | grep -qE "(rapi-doc|stoplight|elements)"
docker compose down -v

# Examples are shellcheck-clean
shellcheck examples/reporters/*.sh examples/consumers/*.sh examples/scheduler/host.crontab 2>/dev/null || true

Handoff

1. Commit:

   feat(M13): polish — OpenAPI, README, doc/, examples, e2e demo
   
   - openapi.yaml served at /api/v1/openapi.yaml; /api/docs viewer
   - README with quickstart, OIDC pointer, scheduler options, backups
   - doc/{architecture,api-overview,auth-flows,frontend-development,api-reference}.md
   - examples/{reporters,consumers,scheduler,reverse-proxy} with shell-checked scripts
   - tests/e2e/demo.sh: clone-to-blocklist in ~10 minutes
   - CI: openapi validation, doc-endpoint accuracy check
   

2. Append to PROGRESS.md:

   ## M13 — Polish, OpenAPI, docs (done)
   
   **Built:** OpenAPI + viewer; README; all five doc files per SPEC §16; examples; e2e test.
   
   **Notes for next milestone:**
   - OpenAPI generation is via [zircote/swagger-php OR hand-curated array]; the source is `<path>`.
   - Doc CI guard: ./scripts/check-doc-endpoints.sh
   - examples/ scripts use IRDB_URL and IRDB_TOKEN env vars; document this convention.
   - The "future user-token flow" in doc/auth-flows.md is the recommended extension point for SPA/native/mobile UIs.
   
   **Deviations from SPEC:** none.
   **Added dependencies:** [zircote/swagger-php if applicable]
   

3. Stop. Do not start M14.