atomic-design-poc/docs/backlog/WP-22-durable-persistence.md

# WP-22 — Durable persistence (optional tier)

Status: todo
Phase: 5 — productie-volwassenheid

## Why

Every backend store (`ApplicationStore`, `DocumentStore`, `BriefStore`) is a
`static Dictionary` guarded by a single `lock` object, explicitly documented as
in-memory ("no DB", per `backend/README.md` and CLAUDE.md's own framing). Data —
including the audit log — is lost on every restart. This is a deliberate POC
simplification (CLAUDE.md lists "runtime DTO validation on every endpoint" and
similar as out-of-scope, and a database was never promised), but it's the one gap
that would visibly break the moment someone tries to run this as a real demo across
multiple sessions or deploys it anywhere that restarts (e.g. most PaaS platforms
recycle instances).

This WP is marked **optional tier** — lower priority than WP-18/19/20/21 — because
unlike auth/e2e/i18n/resilience, the current in-memory design is explicitly
documented and defensible for a POC. Do this when the POC needs to survive restarts
(demoing over multiple days, deploying somewhere with instance recycling), not
speculatively.

## Read first

- `backend/README.md` (the "in-memory seeded, no DB" framing to preserve or
  supersede)
- `backend/src/BigRegister.Api/Data/ApplicationStore.cs`,
  `backend/src/BigRegister.Api/Data/DocumentStore.cs`,
  `backend/src/BigRegister.Api/Data/BriefStore.cs` — the three stores, each
  `static Dictionary` + `lock`
- `backend/src/BigRegister.Api/Data/SeedData.cs` (current in-memory seed — becomes
  a first-run DB seed)
- `docs/architecture/0001-bff-lite-decision-dtos.md` (confirm this WP doesn't touch
  the decision-DTO contracts — persistence is purely behind the existing store
  interfaces)

## Decisions (pre-made, don't relitigate)

- **SQLite + EF Core**, not a heavier database — matches the POC's zero-external-
  infrastructure posture (no docker service to add, no connection string to manage
  beyond a file path) while proving real persistence.
- **Persistence lives entirely behind the existing static-class store APIs** — the
  public methods on `ApplicationStore`/`DocumentStore`/`BriefStore` keep their
  signatures; only the implementation swaps from `Dictionary` to `DbContext`. No
  endpoint or domain-rule code changes (`Program.cs`, `Domain/*`).
- **Seed on empty DB**, not on every startup — `SeedData` runs once (checked via
  "is the DB empty") so restarts don't reset demo data, which is the entire point
  of this WP.
- **Document bytes stay a deliberate exception** if storage size becomes a concern:
  either store them as a BLOB column (simplest, consistent with "one DB, no extra
  infra") or explicitly punt file bytes to disk with only metadata in SQLite —
  decide based on actual seeded file sizes, don't over-engineer a blob-storage
  abstraction for a POC.
- **Audit log becomes a real table**, not just "no longer volatile" — this closes
  the "audit log is in-memory" gap named in the original gap analysis alongside
  persistence, since it's the same static-dict problem in `DocumentStore.cs`.

## Files

- `backend/src/BigRegister.Api/BigRegister.Api.csproj` — add
  `Microsoft.EntityFrameworkCore.Sqlite` + `Microsoft.EntityFrameworkCore.Design`.
- New `backend/src/BigRegister.Api/Data/AppDbContext.cs` — `DbSet`s mirroring the
  three stores' current in-memory shapes (`StoredDocument`, `AuditEntry`, whatever
  `ApplicationStore`/`BriefStore` hold internally — read those files first to avoid
  redesigning the shape, just relocate it).
- `backend/src/BigRegister.Api/Data/ApplicationStore.cs`,
  `DocumentStore.cs`, `BriefStore.cs` — convert static dictionary methods to
  `DbContext`-backed queries; keep every public method signature identical (this is
  the acceptance bar — a signature change means a caller in `Program.cs` or
  `Domain/*` needs to change, which should be zero).
- `backend/src/BigRegister.Api/Data/SeedData.cs` — becomes "seed if empty" run once
  at startup against the real DB.
- `backend/src/BigRegister.Api/Program.cs` — register `AppDbContext` (DI), run
  migrations/`EnsureCreated` + conditional seed at startup.
- New EF Core migration (generated via `dotnet ef migrations add Initial`).
- `.gitignore` — exclude the runtime `.db` file (ship the migration, not the
  database).
- `backend/README.md` — update "in-memory seeded, no DB" framing to describe the
  SQLite file and its lifecycle (created/seeded on first run, persists thereafter,
  delete the file to reset demo data).
- `docker-compose.yml` — mount a volume for the SQLite file so `docker compose up`
  restarts don't lose data either (currently the `api-bin`/`api-obj` volumes exist
  for build caching only, not data).

## Steps

1. Add the EF Core packages; define `AppDbContext` matching the current in-memory
   record shapes exactly (no schema redesign in this WP).
2. Convert one store at a time (`DocumentStore` first — it's the smallest and has
   the audit log, which is the most valuable win), keeping
   `backend/tests/BigRegister.Tests/*` green after each conversion.
3. Wire `AppDbContext` + startup migration/seed in `Program.cs`.
4. Convert `ApplicationStore`, then `BriefStore`.
5. Update `docker-compose.yml` with a persistent volume; update `backend/README.md`.
6. Full backend test suite + a manual restart test: run the backend, create an
   application, restart the process, confirm the application still exists.

## Acceptance criteria

- [ ] All three stores are EF Core/SQLite-backed; no `static Dictionary` remains in
      `Data/*.cs` for application/document/brief state.
- [ ] Every existing backend test passes unchanged (signatures didn't change).
- [ ] Restarting the backend process preserves previously created applications,
      documents, and brief drafts (manually verified).
- [ ] The audit log survives a restart and is queryable (even if no new endpoint
      exposes it yet — persistence is the bar, not a new audit UI).
- [ ] `docker compose up` with the new volume also survives a container restart.

## Verification

`cd backend && dotnet test`. Manual: `dotnet run --project src/BigRegister.Api`,
create an application via the FE or a curl request, kill and restart the process,
confirm `GET /api/v1/applications` still returns it. Repeat with `docker compose up`

- `docker compose restart api`.

## Out of scope

A production-grade database (Postgres/SQL Server) — SQLite is the deliberate,
right-sized choice for a POC that still wants to prove real persistence. Migrating
existing in-memory demo data on upgrade (a fresh SQLite file starts from
`SeedData`, same as today's in-memory start). Blob storage for document bytes
beyond a BLOB column (only revisit if seeded files are large enough to matter).

## Risks

EF Core's async patterns don't drop in as a 1:1 replacement for synchronous
dictionary lookups — endpoint handlers in `Program.cs` currently call store methods
synchronously; converting to `async`/`await` may ripple further than "just the
Data/ layer" if minimal-API handlers aren't already `async`. Check this before
starting and budget for handler signature changes (still not a _behavior_ change,
but a wider diff than the Files section implies if handlers need `async` added).