atomic-design-poc/docs/project/backlog/WP-54-openzaak-integration-harness.md

# WP-54 — Docker OpenZaak integration-test harness

Status: done
Phase: 9 — OpenZaak / ZGW integration

## Why

Everything ZGW so far is verified against **fixtures + a stub `HttpMessageHandler`** — nothing
exercises a real OpenZaak. That is fine for unit-testing the mapper/JWT/pagination, but it does
not prove the BFF actually talks to OpenZaak (auth accepted, real response shapes, real
pagination, zaaktype resolution). This WP stands up a **local OpenZaak via docker**, seeds a
minimal catalog + a client credential matching `ZgwOptions`, and adds an **opt-in** integration
test that points the BFF at it with `Zgw:Enabled=true`. It closes the "no live instance" gap
called out in the roadmap and makes WP-50/51/52 developable against something real.

## Context — current state (read before designing; self-contained handoff)

- **What exists to test**: `backend/src/BigRegister.Api/Zgw/OpenZaakZaakSource.cs` (lists zaken,
  follows `{count,next,previous,results}`, resolves zaaktype labels from Catalogi, Bearer auth
  via `ZgwTokenProvider`), selected when `Zgw:Enabled=true`. Config is the `Zgw` section of
  `appsettings.json` (`ZrcBaseUrl`, `ZtcBaseUrl`, `ClientId`, `Secret`, `UserId`,
  `UserRepresentation`) → `Zgw/ZgwOptions.cs`.
- **Current tests** (the pattern to extend, not replace): `ZgwZaakMapperTests` (inline JSON),
  `OpenZaakZaakSourceTests` (stub handler). These stay as fast unit tests; the new integration
  test is a **separate, opt-in** category so the default `dotnet test` and CI stay fast/offline.
- **Existing compose**: repo root `docker-compose.yml` runs FE + BFF only (Swagger at :5000).
  Do **not** bolt OpenZaak onto it — OpenZaak is heavy (postgres + redis + celery). Use a
  **separate** compose file so `docker compose up` stays light for everyone else.
- **Test project**: `backend/tests/BigRegister.Tests/` (xunit). It uses
  `Microsoft.AspNetCore.Mvc.Testing` (`TestWebApplicationFactory.cs`) — the factory can be
  configured to override the `Zgw` config section to point at the compose instance.
- **CLAUDE.md / backlog GREEN**: the default local gate and CI must remain runnable without
  docker/OpenZaak. Anything requiring the harness is explicitly separate (like e2e is today —
  a distinct job, not chained into `npm run ci`).

OpenZaak facts that shape the harness (from the ZGW research):

- OpenZaak is the reference impl of the 5 ZGW APIs; the published
  `docker-compose.yml` (github.com/open-zaak/open-zaak) brings up **openzaak (web + celery),
  postgres, redis** and (optionally) Open Notificaties. A one-time **bootstrap** is required:
  create a superuser, register an **Application** in the Autorisaties API with a **Client
  ID/Secret + scopes** (`zaken.lezen`, `catalogi.lezen`, …), and import/create a **Catalogus
  with a zaaktype** (published) so there is something to read.
- Auth = the HS256 JWT the BFF already mints; the harness's client credentials must **match**
  `ZgwOptions.ClientId`/`Secret`.
- Reading a zaak needs read scope on **both** Zaken and Catalogi (zaaktype resolution).

## Read first

- [openzaak-integration.md](../reference/openzaak-integration.md) (the seam + config keys).
- [ADR-0005](../reference/architecture/0005-openzaak-behind-bff.md).
- WP-19 (`WP-19-e2e-smoke.md`) — the precedent for a heavy, separate, opt-in test job (mirror
  its "not chained into the default gate" structure).
- Upstream: OpenZaak docker docs (https://open-zaak.readthedocs.io/) + the repo's
  `docker-compose.yml` and its `import` fixtures for a demo catalogus.

## Decisions (pre-made, don't relitigate)

- **Separate compose file** (e.g. `backend/openzaak/docker-compose.openzaak.yml`), never merged
  into the root compose. `docker compose up` stays FE+BFF only.
- **Opt-in test category.** Tag the integration test `[Trait("Category", "Integration")]`;
  exclude it from the default run (`dotnet test --filter Category!=Integration`) and from the
  standard CI jobs. Provide a documented command / optional manual CI job to run it.
- **Bootstrap is scripted, not manual.** A checked-in setup (compose `import` fixture or a small
  bootstrap script) creates the client credentials (matching `ZgwOptions`), a catalogus, and one
  published zaaktype + one zaak — so the test is deterministic and repeatable.
- **Provider stays stubbed** (WP-53) — this WP is about the ZGW round-trip, not real user auth.

## Files

- New: `backend/openzaak/docker-compose.openzaak.yml` + bootstrap fixture/script + a short
  `backend/openzaak/README.md` (how to bring it up, credentials, teardown).
- New: `backend/tests/BigRegister.Tests/OpenZaakIntegrationTests.cs` (Category=Integration):
  configure the BFF (`WebApplicationFactory`) with `Zgw:Enabled=true` + the compose URLs/creds,
  hit `/admin/cases`, assert the seeded zaak comes back mapped.
- Edit: `docs/reference/openzaak-integration.md` (add a "Run against real OpenZaak" section);
  optionally a manual/gated CI job in `.github/workflows/ci.yml` mirroring the e2e job's shape.

## Steps

1. Add the separate compose file bringing up OpenZaak + postgres + redis; pin image versions.
2. Script the bootstrap: superuser, Autorisaties Application (Client ID/Secret = `ZgwOptions`),
   a catalogus + one published zaaktype + one zaak with a natuurlijk-persoon rol (the seeded BSN).
3. Write the opt-in integration test: point the BFF at the compose instance, assert the mapped
   zaak (id = uuid, type = zaaktype label, status) via `/admin/cases`.
4. Document `docker compose -f … up` + the run command; optionally add a manual CI job.

## Acceptance criteria

- [x] `docker compose -f backend/openzaak/docker-compose.openzaak.yml up` yields a reachable
      OpenZaak with the seeded catalogus + zaak, and credentials matching `ZgwOptions`.
- [x] The Category=Integration test passes against it; the BFF returns the seeded zaak mapped to
      `ApplicationSummaryDto` through the real HTTP + JWT path.
- [x] Default `dotnet test` and `npm run ci` are unaffected (integration test excluded, no docker
      needed); `docker compose up` (root) is unchanged.

## Verification

`docker compose -f backend/openzaak/docker-compose.openzaak.yml up -d` →
`./backend/openzaak/bootstrap-catalogus.sh` → `dotnet test --filter Category=Integration` →
green; then teardown. Actually run (not just planned) during this WP — see Deviations below
for what that surfaced.

## Out of scope

Documenten/DRC + Notificaties services in the harness (add when WP-51/52 land), a permanent
always-on CI job (keep it opt-in/manual — OpenZaak startup is slow), performance testing.

## Risks

- OpenZaak startup is slow + resource-heavy → keep it opt-in; a permanent CI job would blow the
  same runner budget WP's storybook cap just fixed.
- ZGW API/version drift vs the pinned image can change response shapes → pin image versions and
  keep the fixture in the repo.
- Bootstrap client scopes must include `catalogi.lezen` or zaaktype resolution 403s — cover in
  the setup script.

## Deviations from the original plan

- **`heeft_alle_autorisaties: true` instead of granular scopes.** The plan called out
  `zaken.lezen`/`catalogi.lezen` specifically; in practice OpenZaak's scripted config
  (`vng_api_common_applicaties_config`, upstream's own documented `setup_configuration` YAML
  mechanism) exposes an all-scopes flag on the one `Applicatie` this harness ever creates. Since
  that application exists for nothing but this throwaway test instance, granular scopes would
  add YAML-schema risk for no real least-privilege benefit — took the simpler, equally-scripted
  option.
- **A live run found a real production bug, not just a harness wrinkle**: `ZgwHttpClient.cs`
  never sent `Content-Crs`/`Accept-Crs` on any ZGW call. Every ZGW write 412s ("Content-Crs
  header ontbreekt") without it — a real OpenZaak enforces this; the stub `HttpMessageHandler`
  every prior Zgw test used never modelled header requirements, so nothing from WP-49/50 caught
  it before now. Fixed in `ZgwHttpClient.cs` alongside the harness (see
  `docs/reference/openzaak-integration.md`) — this is precisely the class of bug this WP exists
  to catch.
- **Publishing a zaaktype needs more seed data than the plan anticipated**: OpenZaak refuses to
  publish a zaaktype with fewer than one resultaattype or fewer than two statustypen (begin +
  eind), and a resultaattype's `selectielijstklasse` must share a `procesType` with the
  zaaktype's own `selectielijstProcestype` — both cross-checked live against the public VNG
  selectielijst API (`selectielijst.openzaak.nl`). `bootstrap-catalogus.sh` seeds all of this;
  see its comments for the exact values used and why.
- **No celery/celery-beat/nginx in the harness**, unlike upstream's own compose — trimmed for a
  faster-booting, single-purpose harness (this test never asserts on notification delivery,
  which is celery's job). `NOTIFICATIONS_DISABLED=true` is required as a consequence: without a
  celery worker, `NotificationsConfig` has no client, and OpenZaak's `notify()` hook otherwise
  raises inside the same DB transaction as the create — turning a missing-worker problem into a
  500 that rolls back the create it was supposed to just notify about.