Compare commits

..
Author SHA1 Message Date
not 6cfcc4cf83 ci(deploy): deploy the stack to Talos on merge to main (closes #175) (#176)
CI / k8s (push) Successful in 6s
CI / unit (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / mutation (push) Canceled after 0s
CI / verify-stack (push) Canceled after 0s
CI / build (push) Canceled after 9s
CI / lint (push) Canceled after 24s
Deploy to Talos / deploy (push) Successful in 5m1s
## What & why

The chart has been deployable by hand since #25 and linted in CI since #168. This makes a
merged PR actually ship it to the Talos VM on the lab server.

`.gitea/workflows/deploy.yaml` runs on a push to `main` (a squash-merged PR) and on manual
dispatch:

1. **Tunnel** — neither the Kubernetes API nor the in-cluster registry is publicly reachable,
   so 6443, 30500 and 30141 are forwarded over the same SSH hop into the Fedora host that the
   Gitea-runner pipeline uses (`ssh -p 6667 user@labs.respellion.tech`).
2. **Images** — `make k8s-images K8S_REGISTRY=localhost:30500`, pushed *through* the tunnel.
3. **Deploy** — `make k8s-reseed TALOS_HOST=… K8S_REGISTRY=<vm-ip>:30500`, pulled by the node
   from its own NodePort.
4. **Roll** — `rollout restart` + `rollout status` on the nine repo deployments.
5. **Smoke** — `GET /openbaar/register` through the openbaar portal.

Three decisions worth the review:

- **One registry, two names.** The push target (`localhost:30500`, the tunnel) and the pull
  target (`<vm-ip>:30500`, the node's own NodePort) address the same store. The pull name has
  to be the one in the node's registry-mirror patch, which is what makes plain HTTP acceptable.
- **`k8s-reseed`, not `k8s-up`.** A Job's pod template is immutable, so a chart change to any
  bootstrap Job would otherwise fail the upgrade with `cannot patch … with kind Job`. The Jobs
  are idempotent by design, so re-running them every deploy is safe and removes that whole
  class of failure. Cost: a few minutes per deploy, and `seed-zaaktype` needs egress from the VM.
- **No re-run of the checks.** PR CI is the merge gate, so `main` is green by construction.
  Deploys **queue** (`cancel-in-progress: false`) — a `helm upgrade` killed half-way leaves the
  release in `pending-upgrade` and has to be unwedged by hand.

Settings on the repo (already added): secrets `TALOS_SSH_KEY` and `TALOS_KUBECONFIG`
(base64, and its `server:` must be `https://127.0.0.1:6443` — Talos puts `127.0.0.1` in the
apiserver cert SANs, so TLS still verifies through the tunnel); variables `TALOS_VM_IP`
(default `192.168.122.173`) and `TALOS_HOST` (default `localhost`).

Closes #175

## Definition of Done

- [x] Linked Gitea issue (above).
- [ ] Failing test committed before the implementation — **n/a**: this is a deployment
      workflow with no unit under test. Its check is the run itself: `rollout status` and the
      public-register smoke both have to pass or the job fails. `make k8s-lint` / `make k8s-drift`
      (#168) already gate the chart it deploys.
- [x] Implementation — one workflow file, no production code touched.
- [x] Conventional Commits referencing the issue (`refs #175`).
- [ ] CI green — awaiting the run on this PR.
- [x] `docker compose up` unaffected — no service, image or compose file is touched.
- [x] Docs updated — `docs/runbooks/kubernetes-talos.md` §9 (the tunnel, the two registry
      names, the secrets table, the smoke) and a pointer from `docs/runbooks/ci.md`.
- [x] No ADR needed: no new dependency (kubectl/helm/crane are already prerequisites of the
      `k8s-*` targets), no service boundary moved, no CLAUDE.md §8 rule bent.
- [ ] Demo note — not user-visible.

## Notes for reviewers

- **The first deploy is the real test.** It cannot be dry-run: the tunnel, the secrets and the
  registry only exist on the lab server. Merging is how we find out; `Pods on failure` dumps
  `get pods,jobs` if it doesn't.
- **Known gap — the portals still aren't browsable.** PKCE needs a secure context, so a
  NodePort on an IP can't serve them (runbook §5); they need `make k8s-portals` or an SSH
  forward. Giving the server a hostname + TLS is the follow-up, and is where `TALOS_HOST`
  stops defaulting to `localhost`.
- **Databases are `emptyDir`.** Any change to a database pod's template wipes it; the
  `k8s-reseed` in the deploy re-runs the bootstrap, so the stack recovers, but submitted
  registrations do not. Persistence is runbook §6.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #176
2026-09-18 13:53:58 +00:00
not 9d7e8e5b65 ci(k8s): gate the Helm chart in CI + a compose↔chart drift check (closes #168) (#171)
CI / k8s (push) Successful in 5s
CI / lint (push) Successful in 1m27s
CI / build (push) Successful in 1m22s
CI / unit (push) Successful in 1m12s
CI / frontend (push) Successful in 2m7s
CI / mutation (push) Successful in 3m9s
CI / verify-stack (push) Successful in 6m20s
## What & why

The Helm chart landed in #167 with two gaps written into ADR-0033: `make k8s-lint` existed
but no CI job ran it, and *"a second deployment description to keep in step with compose —
nothing enforces that today; a drift check belongs in CI (follow-up)"*. Both are closed here.

**`make k8s-drift`** (`infra/helm/check-drift.py`, stdlib only) compares what each stack
actually deploys rather than diffing two files that differ by design: workload names and
resolved container images, taken from `docker compose config --format json` and a rendered
chart. The six differences that exist today are declared in `DEVIATIONS` with the reason
each was forced — the four `*-init` Django services folded into their web pods, and the two
bootstrap Jobs compose runs from the host — so only a *new* difference fails.

**A `k8s` CI job** runs `k8s-lint` then `k8s-drift` on every push and PR. No cluster, no
marketplace action: helm is fetched as the pinned static binary the Talos runbook already
gives developers.

Closes #168

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation — the red commit reports all six
      real differences; the green commit declares them.
- [x] Implementation makes the test pass.
- [x] Conventional Commits referencing the issue (`refs #168`).
- [x] CI green — awaiting the run on this PR (`make k8s-lint` and `make k8s-drift` pass locally).
- [x] `docker compose up` unaffected — no service, image or compose file is touched.
- [x] Docs updated — `docs/runbooks/ci.md` (job table + the one place local and CI now
      differ), `docs/runbooks/kubernetes-talos.md` §7/§"not ported", and ADR-0033's cost note.
- [x] No ADR needed: no new dependency (python stdlib, and helm/docker were already
      prerequisites of the `k8s-*` targets), no boundary moved, no §8 rule bent.
- [x] Not user-visible, so no demo note.

## Notes for reviewers

Verified by hand that both drift classes fail the check, not just that it passes today:

- bumping `OPENZAAK_TAG` in compose alone → reports `openzaak` and `oz-celery` with both
  image strings;
- adding a workload to `values.yaml` alone → reports it by name.

Deliberate limits (there is a `ponytail:` note in the script):

- **Names and images only**, as sets — no per-workload env, ports or volumes. Those differ
  by design in four documented places, so comparing them would mean re-encoding every
  deviation field by field for very little more signal.
- **The three observability workloads are rendered with `enabled=true`** by the check, even
  though both stacks default them off, so their images can't drift unwatched.
- **`k8s-lint`/`k8s-drift` are not in `make ci`**, to avoid making `helm` a hard
  prerequisite for everyone. That is now the only local/CI difference; it's called out in
  `docs/runbooks/ci.md`.

Follow-ups filed while reviewing the chart, not addressed here: #169 (the published docs
omit every ADR after 0010 and all runbooks but `ci.md`) and #170 (the production-posture
ADR #25 asked for — secrets are still plain text in `values.yaml`).Reviewed-on: #171
2026-09-18 13:25:24 +00:00
not 17f1f2f809 docs(nav): publish every ADR and runbook, gated by a nav check (closes #169) (#172)
CI / lint (push) Successful in 1m46s
CI / build (push) Successful in 1m29s
CI / unit (push) Successful in 1m5s
CI / frontend (push) Successful in 1m41s
CI / mutation (push) Successful in 3m30s
CI / verify-stack (push) Successful in 5m50s
## What & why

`docs/` is the source of truth (CLAUDE.md §12), but only pages listed in `mkdocs.yml`'s nav
are published — and mkdocs' own `validation.nav.omitted_files: warn` keeps the build green
while dropping the rest. So the site had quietly stopped at **ADR-0010** and
**`runbooks/ci.md`**: 31 pages, including every ADR from 0011 to 0034, six of the seven
runbooks, and `synthetic-data.md`, existed in the repo and nowhere else.

- `infra/check-docs-nav.py` fails when a page under `docs/` is not in the nav. It runs in
  `make lint`, so the existing CI job gates it — python3 only, no new tooling, and no
  mkdocs install needed to check it.
- The nav now lists all 34 ADRs, all 7 runbooks and `synthetic-data.md`.
- ADR-0033's `Slice:` header said "none yet"; #25 closed it.
- The landing page gained a pointer to the Talos runbook.

Closes #169

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the fix — the red commit lists all 31 missing pages.
- [x] Implementation makes the test pass.
- [x] Conventional Commits referencing the issue (`refs #169`).
- [ ] CI green — awaiting the run on this PR (`python3 infra/check-docs-nav.py` passes locally;
      `make lint` also needs the .NET SDK, which CI has).
- [x] `docker compose up` unaffected — docs and `mkdocs.yml` only, plus one `make lint` line.
- [x] Docs updated — that is the change.
- [x] No ADR needed: no dependency, no boundary, no §8 rule touched.
- [x] Not user-visible, so no demo note.

## Notes for reviewers

- The check is a **substring test**, not a YAML parse (there's a `ponytail:` note in the
  script): a page's path either appears in `mkdocs.yml` or it doesn't. That keeps it
  dependency-free — `mkdocs.yml` can't be read by `yaml.safe_load` anyway, it carries a
  `!!python/name:` tag for the mermaid fence. It does not check that an entry *points at a
  file that exists*; mkdocs' `not_found: warn` covers that direction.
- ADR labels in the nav are shortened by hand (`"ADR-0013: Behandel-portal wiring"`), since
  several H1s are a full sentence.

**Known gap, not fixed here:** CLAUDE.md §12 says the site is "published via a Gitea Actions
workflow to Gitea Pages", and no such workflow exists — `mkdocs build` is never run, by CI or
by any make target. Gitea has no built-in Pages, so publishing needs a decision (a
`gitea-pages` server, an artifact, or a static host) rather than a patch. Worth its own issue
if the published site is actually wanted; until then this PR makes the nav correct for whoever
runs `mkdocs serve`.Reviewed-on: #172
2026-09-18 12:55:20 +00:00
not 1dd8bd4e1b S-24/#25 · Helm chart + Kubernetes deployment, and Caddy for the portals (#166) (#167)
CI / lint (push) Successful in 1m17s
CI / build (push) Successful in 1m12s
CI / unit (push) Successful in 1m26s
CI / frontend (push) Successful in 2m58s
CI / mutation (push) Successful in 9m1s
CI / verify-stack (push) Successful in 8m53s
## What & why

Two changes, made and verified together on a real cluster.

**S-24 / #25 — a Helm chart for the platform.** One chart, `infra/helm/big-reference`,
whose `values.yaml` is a near-literal transcription of `infra/docker-compose.yml`, rendered
by three generic templates (Deployment, Job, Service) over a `workloads` map. Adding a
service is a values edit. `make k8s-lint` renders and schema-checks the whole stack without
a cluster. The issue asked for a *sketch*; this is deployed and verified end to end (see
below), which is more than it asked for — the part it asked for that is **not** here is the
production-posture write-up (HA, secrets, backup), see Known gaps.

**#166 — Caddy replaces nginx in the portals.** nginx resolves a variable `proxy_pass`
upstream itself, using only the `resolver` directive and never `/etc/resolv.conf`'s search
domains. That had cost two workarounds in one script: rewriting the resolver address for
rootless podman, and injecting a full FQDN so the bare `bff` name could resolve on
Kubernetes. Caddy dials per request through the system resolver, so `reverse_proxy
bff:8080` works on every engine unchanged; `apps/portal-nginx-resolver.sh` and the chart's
`BFF_HOST` env are deleted.

Closes #25
Closes #166

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation — twice: the Caddyfile contract test
      before the Caddyfiles, `make k8s-lint` before the chart.
- [x] Implementation makes the test pass.
- [x] Conventional Commits referencing the issues (`refs #25` / `refs #166`).
- [ ] CI green — awaiting the run on this PR (`make k8s-lint`, `dotnet format` and the new
      unit self-check pass locally; the compose e2e and mutation lanes are CI's).
- [ ] `docker compose up` from a fresh clone reaches green health checks within 3 minutes —
      the portal images were rebuilt and verified standalone, but a full `make up` run has
      not been done on this branch. Please confirm in review or let CI's smoke test speak.
- [x] Docs updated — `docs/runbooks/kubernetes-talos.md` (new), `frontend-decisions.md`,
      `demo-script.md`, and the docs that named nginx.
- [x] ADR added — ADR-0033 (chart) and ADR-0034 (Caddy).
- [ ] Demo note in `docs/demo-script.md` — not added: the deployment target is not a
      user-visible slice, and the Caddy swap is invisible to the demo script beyond the
      wording fix included here.

## How it was verified

Brought up from scratch on a single-node Talos v1.14.0 VM (6 vCPU / 10 GB, virtio disk)
under virt-manager: **29 pods ready and four bootstrap Jobs complete in under three
minutes, zero restarts**, using ~4.4 GB of the VM's 10 GB.

- Full Common Ground path: portal Caddy → BFF → domain → Flowable → ACL → OpenZaak +
  Objecten → NRC → event-subscriber → projection → public register (`INGEDIEND`, reference
  matching the submitted registration).
- Werkbak read with an MFA'd medewerker token → 200.
- The browser flow driven with Playwright against `http://localhost:30140`: secure context,
  `crypto.subtle` present, Keycloak form reached, login completed, **no console errors**.
- Routing checked against a stub BFF: SPA fallback serves deep links, each portal proxies
  its own groups, and a portal does *not* proxy a neighbour's group.

## Notes for reviewers

Three bugs this shook out, each fixed at the cause rather than the symptom:

1. **`command` vs `args`.** Compose's `command:` replaces the image CMD; Kubernetes'
   replaces the ENTRYPOINT. Transcribing one to the other broke every upstream image that
   relies on its entrypoint — postgres refused to run as root, Keycloak tried to exec
   `start-dev`. The chart now `fail`s at render time on `command`.
2. **Concurrent migrations.** Both `/setup_configuration.sh` and `/start.sh` run
   `manage.py migrate`; compose serialises them with `depends_on`, Kubernetes has no such
   edge, so the init Job and its web pod raced (`relation "zgw_consumers_service" already
   exists`). The four Django services now do both steps in order in the web pod — which
   also deletes four workloads.
3. **`emptyDir` databases are wiped by any pod-template change.** `make k8s-reseed` now
   also restarts `event-subscriber` and `projection-api`, which create the projection
   schema on start and otherwise keep writing to a schema-less database.

Known gaps / follow-ups:

- **Secrets.** `values.yaml` carries the dev credentials in plain text (`admin/admin`, the
  ZGW client secret, the two Objecten tokens) and the chart has no `Secret` objects. Fine
  for a laptop demo, and exactly what #25's "production posture" ADR should address — I
  suggest a follow-up issue rather than stretching this PR.
- **No CI gate for the chart yet.** `make k8s-lint` exists but is not wired into
  `.gitea/workflows/ci.yaml`, and nothing enforces that the chart and the compose file stay
  in step. Worth a small follow-up.
- **This is two slices in one PR.** They were built and verified together and the diff is
  entangled (the chart was written against Caddy from the start), so splitting now would
  mean re-creating an nginx-shaped chart to throw away. Happy to split if you'd rather.
- **Rebased onto #161** (merged as #165) rather than merged, to keep the history linear.
  One conflict, in the `unit:` target where both branches add a self-check line — resolved
  by keeping both. #161's `infra/host-browser.yml` arrived with
  `/usr/share/nginx/html/config.json` and is fixed to `/usr/share/caddy/` inside the
  `feat(portals)` commit, so no commit on this branch leaves that overlay pointing at a
  path the images no longer have.Reviewed-on: #167
2026-09-10 08:53:58 +00:00
not d6b3f9764f fix(e2e): bound the Playwright run and make a failed login say why (closes #161) (#165)
CI / build (push) Successful in 1m9s
CI / lint (push) Successful in 1m27s
CI / unit (push) Successful in 1m32s
CI / frontend (push) Successful in 3m19s
CI / mutation (push) Successful in 6m24s
CI / verify-stack (push) Successful in 9m33s
## What & why

#161 is really two defects, and the second one is why the first was undiagnosable.

**A wedged suite consumed the job, and took the post-mortem with it.** Nothing bounded the
Playwright run, so CI stopped the job mid-suite — and `if: always()` does not survive that. Run
739's job metadata shows every step after the e2e as a **0-second failure** stamped at the kill:

```
14 failure  09:48:17 -> 10:14:54  Self-service e2e (Playwright …)
15 failure  10:14:54 -> 10:14:54  verify-stack check summary       ← if: always()
16 failure  10:14:54 -> 10:14:54  e2e spec summary                 ← if: always()
17 failure  10:14:54 -> 10:14:54  Dump container logs on failure   ← if: failure()
18 failure  10:14:54 -> 10:14:54  Tear down                        ← if: always()
```

So the per-spec summary, the container-log dump and the teardown never ran, and the log lost
whatever the killed process had buffered — leaving the single `✘` line the issue was filed from.
`globalTimeout` now makes Playwright stop and *report*: the JSON report is written and those steps
still get their turn. (A `timeout-minutes` on the job would have reproduced the same failure, so
there isn't one.) The "~24-minute gap" is that kill, not necessarily a hang — note run 739 shows
`run_attempt: 2`, and `concurrency.cancel-in-progress` kills an in-flight run on any re-run or push.

**A login that never got its form ate the 90-second test timeout.** Playwright actions auto-wait
until the *test* timeout, not `expect.timeout` — so a portal that serves its page but never
bootstraps (its `config.json` fetch or the OIDC discovery behind `authorize()` failed; `main.ts`
only `console.error`s) spent 90s to report `locator.fill: Test timeout of 90000ms exceeded`: the
symptom, not the cause. That is catalogus.spec's 1.8 minutes. Both Keycloak forms are now asserted
visible first, with a 20s budget and a message naming the step that never happened.

Verified against a real blank-bootstrap portal — the beheer image served with a `config.json` that
is not JSON — which fails in **20.2s** with *"the Keycloak login form never appeared — the portal
did not reach Keycloak (check its config.json fetch and the OIDC discovery …)"*.

**And the summary now says why.** The per-spec table (#136) rendered a verdict icon and nothing
else, so even a surviving summary cost a log dive. Failing specs now carry their first error,
flattened for a table cell (ANSI stripped, newlines collapsed, `|` escaped, clipped) — shape
verified against a real @playwright/test 1.61 failing report, with a stdlib assert self-check on
`make unit`.

Closes #161

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation.
- [x] Implementation makes the test pass; refactor commit follows (login helper dedup).
- [x] Conventional Commits referencing the issue (`refs #161`).
- [ ] CI green — all Gitea Actions jobs.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (untouched).
- [x] Docs updated — `docs/runbooks/gitea-actions-gotchas.md` §9.
- [x] ADR — not needed: no boundary, dependency or coupling rule touched (test/CI infra only).
- [x] Demo note — not applicable: nothing user-visible.

## Notes for reviewers

**What this does not do: identify why the beheerder login failed that once.** The evidence to do
that was destroyed by defect 2, which is what this PR fixes. The suite ran green here five times
today (catalogus.spec 1.1–5.3s each) — but a local box is not the loaded CI runner, so that is weak
evidence and I am not claiming the flake is gone. What changes is that the next occurrence is
bounded and self-describing: it fails in 20s naming the failing step, the JSON report survives, and
the summary prints the error. Please keep #161 in mind rather than treating this as proof.

**Two follow-ups I did not pull into this PR:**
- *All four portals show a permanently blank page if their startup fetch fails* — `main.ts` does
  `fetch('config.json').then(bootstrap).catch(console.error)`, one shot, no UI and no recovery. That
  is a real product gap (the deliberately-broken portal above is exactly what a user would see) and
  wants its own slice, not a test-infra PR.
- `retries: 1` is untouched. CLAUDE.md §15 says flaky tests are fixed rather than retried, but
  removing retries while a real flake is unexplained would trade a rare red for a frequent one.
  Worth revisiting once #161 recurs (or doesn't) with the new diagnostics.

The login-helper rename (`medewerker-login.ts` → `keycloak-login.ts`, citizen logins routed through
`loginBurger`) is its own no-behaviour-change commit: the three citizen specs each duplicated the
same three-line login, so guarding the login path once meant routing them through it first.Reviewed-on: #165
2026-09-04 10:53:35 +00:00
not 8b206a005f S-26/#162 · Werkbak refreshes itself when a registration is ready for beoordeling (#164)
CI / build (push) Successful in 1m8s
CI / lint (push) Successful in 1m23s
CI / unit (push) Successful in 1m27s
CI / frontend (push) Successful in 3m8s
CI / mutation (push) Successful in 6m13s
CI / verify-stack (push) Successful in 10m12s
## What & why

The behandel werkbak now **refreshes itself** while it is open, so a registration that reaches
beoordeling after the behandelaar opened the page shows up on its own — no reload.

`interval(WERKBAK_REFRESH_MS)` (5 s) re-reads the existing BFF endpoint, scoped to the page with
`takeUntilDestroyed()`. A *background* read leaves the rows and states on screen alone until it has
an answer, so a tick never flashes the loading state over rows being read and one failed poll never
swaps the list for the error alert; a read that comes back also clears an earlier failure, so the
view recovers on its own rather than needing the very reload this slice removes.

No new endpoint, dependency or server-side state, and no service boundary moves — rxjs and
`GET /behandel/werkbak` are both already here. **ADR-0032** records why polling rather than a pushed
stream: nothing notifies the BFF either, so SSE/WebSockets would poll the domain *inside* the BFF for
the same freshness, plus connection lifecycle, nginx buffering and a stateful BFF. Proposal: #163.

Closes #162

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation.
- [x] Implementation makes the test pass; refactor commit if structure improved.
- [x] Conventional Commits referencing the issue (`refs #162`).
- [ ] CI green — all Gitea Actions jobs.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (unchanged; only the behandel bundle differs).
- [x] Docs updated if behaviour, contracts, or operations changed.
- [x] ADR added in `docs/architecture/` (ADR-0032).
- [x] Demo note in `docs/demo-script.md` (user-visible).

## Notes for reviewers

**The e2e is the real acceptance test, and it took two goes to make it one.** Simply dropping the
`staff.reload()` from the happy path proved nothing: the werkbak was visited *after* the documents
were supplied, so the row was already there at page load. The spec now logs the behandelaar in
**first**, asserts the row is not there yet, and only then has the citizen supply the documents that
route it to Beoordelen — so the row can only reach that already-open, never-reloaded page via the
refresh. Verified both ways against a live stack: with the interval stubbed out it fails at
`Goedkeuren <ref> … element(s) not found` after 30 s; with it, the behandel nginx logs the poll that
delivers the row. The page is foregrounded before the assertion because Chromium throttles timers in
a hidden tab.

**Ceiling (named in the ADR):** a fixed 5 s interval, per open page, that keeps polling in a
background tab; each tick costs one Flowable task query plus a store read per open task. Upgrade
path: publish task events from the domain, then swap the `interval` for a stream — the endpoint
contract and the rendering stay put. Gate on `document.visibilityState` first if request volume is
the concern.

**Two housekeeping notes, neither blocking:**
- #162 is on **no milestone** (DoD item 1). It is portal UX, so it fits neither *Data Governance*
  nor *Production Posture* cleanly — your call where it lands.
- The issue titles itself **S-26**, which already belongs to the self-service resume slice (#111,
  `BACKLOG.md`). Everything here references **#162**; worth renumbering the title if the S-ids are
  meant to stay unique. `BACKLOG.md` is untouched for the same reason (it mirrors the active
  milestone, and this slice is on none).Reviewed-on: #164
2026-09-04 09:34:14 +00:00
not d0fb2b3e8c S-15c · Enforce MFA on the medewerker (Keycloak) realm (#158)
CI / build (push) Successful in 1m7s
CI / lint (push) Successful in 1m22s
CI / unit (push) Successful in 1m24s
CI / frontend (push) Successful in 3m5s
CI / mutation (push) Successful in 6m13s
CI / verify-stack (push) Successful in 8m39s
Closes #132.

Staff logins (behandel + beheer portals) now need a second factor; the citizen realms are unchanged.

**How:** every seeded medewerker carries a TOTP credential, which activates Keycloak's stock *conditional OTP* step in both the browser flow and the direct grant — no custom browser-flow JSON in the export. `CONFIGURE_TOTP` is a default required action so a medewerker added later must enrol first. ADR-0031 records the choice and, explicitly, that the shared fixture secret is a demo posture only.

**Tests (red first, 30c5279):**
- `check_realms.py` asserts the medewerker password-only grant is **refused**, then that password + TOTP succeeds and still carries the `behandelaar` role. It failed with `[MFA NOT ENFORCED]` against the old export.
- The three medewerker e2e logins move to `loginMedewerker()` (`tests/e2e/medewerker-login.ts`), which submits Keycloak's OTP prompt. Both TOTP implementations (Python `hmac`, Node `crypto`) are ~6 lines of RFC 6238 — no new dependency.

Verified locally against Keycloak 26.1: password-only → `invalid_grant`, password + code → 200, and the browser flow's `#otp` prompt accepts a computed code and issues an auth code.

## Definition of Done
- [x] Failing test/verify committed first; implementation makes it pass.
- [x] Conventional Commits referencing the issue (`refs #132`).
- [ ] CI green (verify-stack compose smoke + relevant checks).
- [x] `docker compose up` reaches green health within 3 minutes (Keycloak change is import-time only).
- [x] Docs touched (runbook, synthetic-data, demo-script) + ADR-0031 + demo note.
- [x] Closed by the merging PR (`closes #132`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #158
2026-09-04 08:27:52 +00:00
eho 321ee50dcb docs(architecture): import the FDS architecture decisions from the lab repo (closes #159) (#160)
CI / build (push) Successful in 1m36s
CI / lint (push) Successful in 1m45s
CI / unit (push) Successful in 2m1s
CI / frontend (push) Successful in 3m30s
CI / mutation (push) Successful in 8m4s
CI / verify-stack (push) Successful in 9m25s
## What & why

Brings the engineer-facing FDS documentation next to the code it describes. Imported from `projects/open-register-fd/` in `Respellion/innovation-lab` and translated to Dutch: **six ADRs**, the ADR index and template, the **L3 component view**, and the **slice-1 proposal**.

The architecture blueprint, the FDS gap analysis and the two privacy views stay in the lab repo — the OKRs cite them and they feed tender responses. Each side names the split in a "Wat ligt waar" table, so nothing is documented twice.

Closes #159

### Why `docs/architecture/fds/` and not `docs/architecture/`

This repo's own ADR series now runs `adr-0001-loose-coupling` … `adr-0010-bff-oidc`. The imported set is numbered 0001–0006, so a flat import would collide across the whole imported range. The subfolder preserves the imported numbering, and with it roughly thirty `ADR-000N` cross-references inside the imported text that would otherwise all need rewriting.

In the MkDocs sidebar the imported six appear as **FDS ADR-000N** so they are not confused with this repo's series. `docs/architecture/fds/README.md` explains the two series.

### Mermaid support was missing

`pymdownx.superfences` had no `custom_fences`, so the imported diagrams would have published to Gitea Pages as raw code blocks. This PR adds the mermaid custom fence, the nav group, and one link under *Where to go* in the docs index.

## Definition of Done

- [x] Linked Gitea issue (above).
- [ ] Failing test committed before the implementation. — n/a, documentation only.
- [ ] Implementation makes the test pass. — n/a, documentation only.
- [x] Conventional Commits referencing the issue (`refs #159`).
- [x] Rebased on current `main`; no conflicts.
- [ ] CI green — n/a for content; the docs verification is below.
- [ ] `docker compose up` reaches green health checks. — n/a, no runtime change.
- [x] Docs updated if behaviour, contracts, or operations changed.
- [x] ADR added in `docs/architecture/` if a non-obvious decision was made. — six imported, plus the numbering decision recorded in the folder README.
- [ ] Demo note in `docs/demo-script.md`. — n/a, nothing user-visible.

## Verification run

- `mkdocs build` — clean. No missing-nav warning for any `architecture/fds/` entry. The two remaining warnings are pre-existing on `main` and untouched here: the set of pages absent from `nav`, and a broken link in `runbooks/ci.md` to `services/acl/stryker-config.json`.
- Mermaid renders as a diagram, not a code block: `site/architecture/fds/c4-component-view/index.html` contains `class="mermaid"`.
- All relative markdown links in the repo resolve.

## Notes for reviewers

- **Language.** The imported documents are Dutch; this repo's own documents remain English. Deliberate, not an oversight — the lab repo standardised on Dutch and these pages moved with it. Translating the rest is a separate decision.
- **Ownership.** This repo sits in the `eho/` namespace while it now holds the canonical FDS architecture decisions that tender answers point at. Worth deciding whether it should move to `Respellion/`.
- **Scope drift, not fixed here.** The imported text is faithful to its source, so the slice-1 proposal and the ADRs assume NHR/KVK for slice 1, while the lab-side blueprint still uses BAG as its example register. The lab-side documents carry a banner about this; Blueprint v2 (slice 5) is where the diagrams get corrected.
- **Companion PR:** `Respellion/innovation-lab` #34 holds the lab-side half of this split.Reviewed-on: #160
2026-09-03 12:35:01 +00:00
not 94720f0fcb fix(observability): stop single-binary Tempo evicting its only ingester (closes #156) (#157)
CI / build (push) Successful in 1m3s
CI / lint (push) Successful in 1m21s
CI / unit (push) Successful in 1m23s
CI / frontend (push) Successful in 3m11s
CI / mutation (push) Successful in 6m19s
CI / verify-stack (push) Successful in 9m57s
## What & why

`verify-tracing` flaked on `verify-stack` run 722 — `FAIL — no single trace spanned ['bff', 'projection-api']` — and went green on a plain re-run of the same commit. **The trace chain was not broken; Tempo could not ingest:**

```
removing distributor_pool failing healthcheck addr=127.0.0.1:9095
  reason="rpc error: code = DeadlineExceeded"
pusher failed to consume trace data  err="context canceled"   (x18)
```

The root cause is the *mechanism* of the data loss, not whatever caused the stall. Tempo runs **single-binary**, so the distributor and the ingester are the same process and the distributor's ingester pool holds exactly one, in-process, member. dskit nevertheless health-checks that member over loopback gRPC with a **1 s** deadline (`checkinterval: 15s`, confirmed from the running image's `/status/config`). On the shared runner a transient stall blows the deadline, the only ingester is evicted from the pool, and every subsequent push fails until the next check interval — spans silently dropped.

With one in-process ingester the health check can **never** route around a failure. Its only possible effect is to discard data. So it is off:

```yaml
ingester_client:
  pool_config:
    healthcheckenabled: false
```

This lands at the point where *both* candidate triggers named in #156 (GC pressure near `mem_limit`, CPU contention from the grown stack) turn into lost spans, so **`mem_limit: 400m` is untouched** — raising it on a memory-tight runner risks reintroducing the `verify-e2e` OOM of #144. It also does not paper over anything the way a longer `TRACING_TIMEOUT` would (#156's own note).

Second change: `infra/tracing-check.py` prints `tempo_distributor_ingester_clients` on its failure path. From the check's side, Tempo-dropped-spans and missing instrumentation look identical — that ambiguity is what cost a container-log dive on run 722. A recurrence now names itself.

Closes #156

## Definition of Done

- [x] Linked Gitea issue (#156).
- [ ] **Failing test committed before the implementation — N/A, and deliberately so.** The trigger is runner load, so no deterministic red exists; the "red" is run 722's observed `verify-tracing` failure plus its Tempo logs. Same precedent as d5e5fa2 (#115, Playwright OOM) and 4aafd32 (#147, uWSGI caps). A test asserting the config says what the config says would add no gate: Tempo hard-fails on an unknown key (verified — `field health_check_enabled not found in type client.PoolConfig`), so a typo or a config rename on a Tempo bump already turns `verify-up` red.
- [x] Conventional Commits referencing the issue (`refs #156`).
- [ ] CI green — the point of the change.
- [x] `docker compose up` health unaffected (Tempo is not in `WAIT_SVCS`; config-only change, same image).
- [x] Docs updated — ADR-0023 Consequences.
- [x] ADR — amended **ADR-0023** rather than adding a new one: this is a consequence of that ADR's single-binary Tempo choice, not a new decision (one decision per ADR, §12).
- [x] Demo note — N/A, not user-visible.

## Notes for reviewers

Verified locally against the built image (the flake itself is not locally reproducible — see the runner-load point above):

1. `docker run --rm register-referentie/tempo:dev -config.file=/etc/tempo.yaml -config.verify=true` → parses.
2. `GET /status/config` on the running container → `healthcheckenabled: false` (was `true`).
3. The new diagnostic reads `tempo_distributor_ingester_clients` off a live Tempo.

Worth knowing: that metric is legitimately `0` on an idle Tempo — the pool is populated lazily on first push. It only prints on the failure path of a check that has already generated traffic, so the reading is meaningful there, but don't read a bare `0` on a quiet stack as an eviction.

Follow-up left undone: if `verify-tracing` still flakes after this, the next suspect is the .NET OTLP exporter timeout (#156's last note), not Tempo's memory cap.Reviewed-on: #157
2026-09-01 08:31:36 +00:00
not 94742a261f feat: read projection sourced from the register in Objecten (closes #153) (#155)
CI / build (push) Successful in 1m7s
CI / lint (push) Successful in 1m26s
CI / unit (push) Successful in 1m37s
CI / frontend (push) Successful in 3m36s
CI / mutation (push) Successful in 6m42s
CI / verify-stack (push) Failing after 11m26s
## What & why

S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the
`RegisterRecord` in Objecten, not from ZGW zaak events.**

Until now the subscriber listened on `zaken` and *inferred* register state from case events — a
`zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not
read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification
at all, so every projection made a second hop to the ACL. The register — a fact about a person —
was being reconstructed by guessing at the lifecycle of the case that produced it.

- The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish).
- An Objecten notification carries **no record data**, only the object URL, so the record is read
  back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as
  ADR-0028 established.
- The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`,
  `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all
  gone, and so is the ACL enrichment hop.
- **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently
  drop every submitted registration from the public register, since only approval wrote a record.
- `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead
  of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all.

**ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must
agree" — is closed: there is one source now.

Closes #153

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing tests committed before the implementation — two red/green pairs, ACL side
      (06c0444566ef7d) and subscriber side (142ed458af09b2).
- [x] Refactor commit follows (b496ac9).
- [x] Conventional Commits referencing the issue (`refs #153`).
- [x] CI green — all six jobs on b30fa66, `verify-stack` end to end including the e2e.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step — see the wait-healthy fix below).
- [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md,
      e2e header comment.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register
      shows the same two statuses for the same registrations; only where they come from changed.

## Notes for reviewers

**The decision I'd most like a second opinion on** is the one the issue didn't settle: what
happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice
between (a) the ACL also writing on submit, (b) a public register that lists only actual
registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged
and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public
register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with
diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling
ADR-0028 set out to remove. All three are laid out in ADR-0030.

**The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the
object URL (the ACL upserts *one object per registration*, so submit and approval notify about
the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried
approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6.

**The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` →
`register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning
something else, and a rebuild would then have projected that garbage. It also empties both
tables: a pre-slice row describes a zaak event the new projector can't reproject, and those
registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new
source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten
if a long-lived environment ever needs it.

**`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against
OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back
produces no projection row — that's the re-source working, not a gap.

## Three fixes CI found, none of them in the projection logic

1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with
   `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed
   objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also
   matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker
   listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those
   services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db`
   the same way. Anchored on the compose replica suffix, which the verify scripts already do.
2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL
   put this check in the same bind run-domain-check.sh already handles:
   `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak
   reflects the request Host into the zaaktype URL and then rejects it on zaak-create when
   single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`.
3. **Approval arrives as `partial_update`, not `update`** (0dd26a7b30fa66) — the one real bug
   in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but
   names the action `partial_update`, so the projector dropped every approval. Only the e2e could
   catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that
   drives a *real* approval.

`verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a
plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` /
`distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace
chain being broken. Filed as **#156** rather than absorbed here.

**Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if
verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin
image, so `objecten-celery` is already a single-process worker. Noted in #156.

**Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring
`objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and
run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too
broad a ripple to land inside an unrelated slice — worth its own issue.

**Known costs, all in the ADR:** submission is now two writes across two modules and eventually
consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being
reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak
still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc`
asserts that path.Reviewed-on: #155
2026-09-01 07:26:33 +00:00
not 2125fb0cfd feat(infra): Objecten publishes register events to NRC (closes #152) (#154)
CI / build (push) Successful in 1m14s
CI / lint (push) Successful in 1m28s
CI / unit (push) Successful in 1m28s
CI / frontend (push) Successful in 3m8s
CI / mutation (push) Successful in 6m35s
CI / verify-stack (push) Successful in 9m27s
## What & why

S-19b-1. A write to the Objecten API now produces a **delivered** notification on the
`objecten` kanaal in Open Notificaties. ADR-0028 switched Objecten's notifications off on
purpose — there was no broker, worker, kanaal or abonnement, so wiring only the client side
would have dropped every message on the floor. This slice builds the real path and turns it
back on.

- `objecten-celery` worker (mirrors `oz-celery`) + `CELERY_BROKER_URL`/`RESULT_BACKEND` on
  objecten-redis db 1 (db 0 is already the cache). `notifications_api_common` only *queues*
  the send; without a worker every register write is silently undelivered.
- `nrc` service + `notifications_config` in Objecten's `setup_configuration`, reusing the
  `big-reference-seed` credential OpenZaak publishes with (NRC authorizes it via OpenZaak's
  AC, which grants it `heeft_alle_autorisaties` — no second credential needed).
- The `objecten` kanaal in NRC's `setup_configuration`. The name is fixed by the Objects API
  (`NOTIFICATIONS_KANAAL`), not chosen here; publishing to an unregistered kanaal is exactly
  what the red check reported first.
- `NOTIFICATIONS_DISABLED: "false"` in both compose files.
- Writers address Objecten as `objecten.local` — see *Notes for reviewers*.
- `make verify-objecten-notifications` — registers an abonnement on `objecten` pointing at a
  throwaway sink, writes a `RegisterRecord` exactly as the ACL does on approval, asserts the
  delivery. One assertion covering the whole chain: Objecten -> objecten-celery -> NRC ->
  nrc-beat -> callback. Wired into the CI `verify-stack` job and the summary table.

**ADR-0029** records the decisions; ADR-0028's ceiling now points at it.

Closes #152

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation (dc9ca2c, red at the first hop:
      `NRC POST /api/v1/abonnement -> 400 "Kanaal met deze naam bestaat niet."`).
- [x] Implementation makes the test pass (4488962, + two fixes found by CI, below).
- [x] Conventional Commits referencing the issue (`refs #152`).
- [x] CI green — all six jobs on a5fd47e, including `verify-stack` end to end (e2e included).
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step).
- [x] Docs updated — ADR-0029 added, ADR-0028's ceiling annotated, BACKLOG.md split.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` if user-visible — n/a, infrastructure only; nothing
      consumes the kanaal until S-19b-2 (#153).

## Notes for reviewers

**The one genuinely non-obvious bit: writers address Objecten as `objecten.local:8000`, not
`objecten:8000`.** NRC types a notification's `hoofdObject`/`resourceUrl` as DRF `URLField`,
so Django's `URLValidator` runs on them — and it rejects a **single-label** host. Objecten
fills both from the object url DRF built with `request.build_absolute_uri`, i.e. *the Host
the caller used*. Writing via the plain service name returns 201 and then fails every
publish in the background, forever, with

```
400 {"hoofdObject":["Voer een geldige URL in."],"resourceUrl":["Voer een geldige URL in."]}
```

So the `objecten` service carries an `objecten.local` network alias and every writer uses it
— `Acl__Objecten__BaseUrl`, `ObjectenGatewayIntegrationTests`, this slice's verify driver.
An alias rather than a bare dotted `SITE_DOMAIN` so the host still *resolves*: a subscriber
following `resourceUrl` reaches the record, which S-19b-2 will do. Readers keep the plain
name. Same class of constraint as ADR-0028's Objecttypen base-URL rule.

**Ceiling, stated in the ADR:** nothing enforces the alias — a future writer using
`objecten:8000` gets a 201 and silently no notification. If a second writer ever appears,
rename the compose service rather than adding a lint.

**Two CI-only failures on the way here**, both worth knowing:
1. `SITE_DOMAIN` was my first guess at the mechanism and is simply not what builds those
   URLs — dropped in d76abf2.
2. The check correlated the delivery on the `reference` inside the record it wrote. An NRC
   notification carries `kanaal`/`resource`/`kenmerken`/`hoofdObject`/`resourceUrl` and
   **never the record data**, so it correlates on the object URL now (a5fd47e).

**Cost:** one more long-running container on the memory-tight runner. It inherits the capped
`UWSGI_PROCESSES: "1"` env, which the celery command ignores; if `verify-stack` gets tight
again, celery concurrency is the next knob.

**Follow-up:** S-19b-2 (#153) sources the projection from these events. Nothing subscribes to
the `objecten` kanaal in the product yet — only the verify check does.Reviewed-on: #154
2026-08-28 10:11:54 +00:00
not 0cd70ae8c3 S-19a · ACL writes the RegisterRecord to Objecten on approval (closes #149) (#151)
CI / build (push) Successful in 1m8s
CI / lint (push) Successful in 1m23s
CI / unit (push) Successful in 1m22s
CI / frontend (push) Successful in 3m3s
CI / mutation (push) Successful in 6m22s
CI / verify-stack (push) Successful in 7m54s
Closes #149.

**Outcome:** approving a registration now writes the canonical register record to the **Objecten** API as a `RegisterRecord` object, alongside the ZGW eindstatus. OpenZaak holds the process, Objecten holds the register (ADR-0028). The write goes through the ACL (§8.1) and is idempotent on the zaak id, so a replayed approval updates the existing object rather than creating a second one.

S-19 (#20) was split first (CLAUDE.md §13) — it bundled this with re-sourcing the read projection, which is now #150.

### What landed

- `IRegisterRecordGateway` + `RegisterRecord` in `Acl.Application`; `ObjectenGateway` in `Acl.Infrastructure` (static Token auth, CRS headers, objecttype resolved by name to its highest **published** version).
- `AclService.ApproveZaakAsync` writes the record after the eindstatus, keyed on the zaak UUID with the zaak's identificatie as reference.
- Compose wiring for both stacks; `ADR-0028`; demo note; PRD §15 out-of-scope line retired.

### Three things only a live stack found

Running the gateway against a real Objecten + Objecttypen pair while writing this turned up blockers CI would have hit after the fact:

1. **Objecten rejects an objecttype it has not been configured with**, by UUID — assigned at seed time by a one-shot that runs *after* Objecten's static setup_configuration. The UUID is now pinned on both sides.
2. **Objecten 500s on every write when its Notificaties config is absent** (`notifications_api_common` raises rather than skipping). Objecten → NRC has no broker, worker, kanaal or abonnement, so notifications are **disabled** rather than wired to drop every message; #150 turns them on for real.
3. **Objecttypen echoes the request Host into the objecttype `url`**, and Objecten only accepts the one matching its configured `api_root` — so the ACL must read Objecttypen at `http://objecttypen:8000`. This is why the new integration test only passes inside the compose network.

All three are recorded in ADR-0028.

### Verification

- `ObjectenGatewayIntegrationTests` (verify-acl, in-network): two writes for one id leave exactly one object with the second write's status. **Passing locally against live Objecten.**
- The **Playwright happy path** asserts, after the behandelaar approves, that Objecten holds exactly one `RegisterRecord` for *that* reference — missing, duplicated, or non-public-safe all fail.
- ACL mutation score **92.23%** (baseline 91.37%, break 90).
- `make lint` / `make unit` green locally; full-stack `make verify` runs in CI.

## Definition of Done

- [x] A linked Gitea issue exists (#149).
- [x] Failing test written and committed first.
- [x] Implementation makes the test pass.
- [x] Refactor commit follows if structure improved.
- [x] Conventional Commit messages referencing the issue (`refs #149`).
- [x] All Gitea Actions CI jobs green (run 684).
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (verify-stack step 1).
- [x] Docs touched — ADR-0028, demo note, PRD §15, BACKLOG.
- [x] ADR added: `docs/architecture/adr-0028-objecten-holds-the-register.md`.
- [x] Demo note appended to `docs/demo-script.md`.
- [x] Closed by the merging PR (`closes #149`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #151
2026-08-14 09:34:04 +00:00
not d37d4c96c6 S-18c · RegisterRecord objecttype defined + registered (closes #141) (#146)
CI / build (push) Successful in 1m29s
CI / lint (push) Successful in 1m37s
CI / unit (push) Successful in 2m13s
CI / frontend (push) Successful in 4m31s
CI / mutation (push) Successful in 7m0s
CI / verify-stack (push) Successful in 8m52s
## What & why

S-18c, the **final** slice of the S-18 (#19) split (after S-18a #142, S-18b #143). Defines the **RegisterRecord** objecttype — the schema S-19 (#20) will write canonical register records against on approval — and registers it in the Objecttypen API at startup.

Closes #141

### What

- **Schema** (`infra/objecttypen-registerrecord/registerrecord.schema.json`): public-safe by construction — `id`, `status` (enum `INGEDIEND`/`INGESCHREVEN`), `reference` only, `additionalProperties: false`, `dataClassification: open`. Mirrors the BFF's `OpenbaarEntry` — **no `bsn`/`naam`** (ADR-0027).
- **Registration**: a `registerrecord-init` compose one-shot (stdlib Python on the stack network) POSTs the objecttype + a **published** version over the API once Objecttypen is healthy. The Objecttypen `setup_configuration` (3.4.2) only provisions tokens — no declarative objecttype step — so this follows the ADR-0020 self-seed pattern. **Idempotent**: if a `RegisterRecord` with a version already exists it is a no-op.
- **Wiring**: schema + `register.py` streamed into the external `rr-registerrecord-config` volume by `seed-config.sh registerrecord` (main) / bind-mounted (local); added to `SEED`, `CFG_VOLS`, and the CI log-dump. `registerrecord-init` is a one-shot (not in `WAIT_SVCS`).
- **Smoke**: `verify-registerrecord` (`run-registerrecord-check.sh` + `registerrecord-check.py`) asserts the objecttype exists, has a **published** version, and that version's schema carries `id`/`status`/`reference`; added as a verify-stack step + a row in the #136 summary.
- **ADR-0027**: records the public-safe schema decision (mirror the BFF public view, not the internal projection; API-seeded one-shot). The slice issue #141 flagged the schema as ADR-worthy, so no separate adr-proposal issue was opened.

## Verified locally (end to end, real compose)

Seeded `rr-registerrecord-config`, brought Objecttypen up, ran `registerrecord-init` → `registered RegisterRecord <uuid> v1 (published)`. `make verify-registerrecord` → **OK — RegisterRecord v1 published, fields=['id', 'reference', 'status']**. Re-running the one-shot → **no-op** (idempotent). `docker compose config` clean on both files; schema + script + ci.yaml validated.

## Definition of Done

- [x] Failing smoke committed first (`test(infra): …`, "no objecttype named RegisterRecord"); implementation makes it pass.
- [x] Conventional Commits referencing #141.
- [x] CI green (verify-stack registerrecord step — validated locally; runner already unstarved by #145).
- [x] `docker compose up` reaches health (one-shot registers after Objecttypen healthy).
- [x] Docs: ADR-0027 + demo note.
- [x] Closed by the merging PR (`closes #141`).

This closes out the S-18 (#19) split — Objecttypen (S-18a) + Objecten (S-18b) + RegisterRecord (S-18c) are all up. Next: **S-19 (#20)** — ACL writes the register record to Objecten on approval, against this schema.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #146
2026-07-27 15:14:16 +00:00
not 159f014c1e perf: cap OpenZaak + NRC uWSGI workers — shrink verify-stack footprint (closes #147) (#148)
CI / build (push) Successful in 1m50s
CI / lint (push) Successful in 1m54s
CI / unit (push) Successful in 2m14s
CI / frontend (push) Successful in 4m45s
CI / mutation (push) Successful in 10m9s
CI / verify-stack (push) Failing after 11m19s
## What & why

Closes #147. Follow-up to #144/#145. As the stack grew to **37 services** on one runner, `verify-stack` is under memory pressure. #145 capped Objecten/Objecttypen; this caps the two biggest remaining uncapped Django apps.

**OpenZaak** and **NRC** (`nrc-web`) are Maykin/vng uWSGI images running the image default of **4 processes × 4 threads** — ~4 full-Django worker processes (~800 MB) each, idle, serving only single-request smoke checks.

### What

- `UWSGI_PROCESSES: "1"` + `UWSGI_THREADS: "2"` on the `&oz-env` and `&nrc-env` anchors, in both compose files. Frees ~1.2 GB. The anchors are shared with the `-init` (setup_configuration) and `-celery` containers, which ignore the var — they don't run uwsgi.

### Not included (considered, deferred to #147 notes)

JVM heap caps on Keycloak/Flowable; compose profiles to boot per-check subsets.

## Verified locally

OpenZaak brought up healthy with the cap; uwsgi processes **6 → 3** (master + http-router + 1 worker); `/admin/` still 302. `docker compose config` clean on both files. (Full NRC bring-up needs OpenZaak + the seed chain — same image family/lever, validated via OpenZaak.)

## Definition of Done

- [x] Linked issue (#147).
- [x] Conventional Commit referencing #147.
- [x] Verified locally (OpenZaak healthy + worker count dropped + still serving).
- [x] Closed by the merging PR (`closes #147`).

No ADR: config-only tuning of existing services, same class as #145.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #148
2026-07-27 14:32:13 +00:00
not dd54688f86 fix: cap Objecten/Objecttypen uWSGI to 1 worker — unstarve verify-stack e2e (closes #144) (#145)
CI / build (push) Successful in 1m32s
CI / lint (push) Successful in 1m41s
CI / unit (push) Successful in 2m21s
CI / frontend (push) Successful in 4m56s
CI / mutation (push) Successful in 7m54s
CI / verify-stack (push) Failing after 16m47s
## What & why

**P0 — red `main`.** Fixes #144: `verify-stack` fails on the Playwright e2e step (main runs 2177 after #142, 2190 after #143), while the PR runs passed.

Closes #144

### Root cause

The Maykin **Objecttypen** (S-18a) and **Objecten** (S-18b) images run their `web` under uWSGI with **4 processes × 4 threads by default** (`UWSGI_PROCESSES:-4`). Two web services × 4 idle Django workers (~200 MB each) sat idle during the e2e step and starved the single shared self-hosted runner — Keycloak and the portals stopped responding (the login `#username` never appeared) and Chromium hit `Target crashed`. The runner margin was already thin; the second chain tipped it over (main green through run 2159, red from 2177).

### Fix

Cap `UWSGI_PROCESSES: "1"` + `UWSGI_THREADS: "2"` on both `objecten` and `objecttypen` in both compose files. These APIs only serve single-request smoke checks and are idle during e2e, so 1 worker is plenty — it frees ~1–1.5 GB. The `-init` containers ignore it (they run `setup_configuration`, not uwsgi).

## Verified locally

Brought the objecten chain up with the cap: both services reach healthy, worker count drops from 6 (master + http + 4 workers) to 3 (master + http + 1 worker) per service, and `make verify-objecten` / `make verify-objecttypen` both still → **OK — no-auth 401, token 200**. `docker compose config` clean on both files.

## Definition of Done

- [x] Linked issue (#144).
- [x] Conventional Commit referencing #144.
- [x] Verified locally (both APIs healthy + smoke green with 1 worker).
- [x] Closed by the merging PR (`closes #144`).

No ADR: config-only tuning of existing services — no boundary, dependency, or coupling change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #145
2026-07-27 13:07:55 +00:00
not 0a97fa4bf7 S-18b · Objecten API up in compose, wired to Objecttypen (closes #140) (#143)
CI / build (push) Successful in 1m29s
CI / lint (push) Successful in 1m45s
CI / unit (push) Successful in 2m6s
CI / frontend (push) Successful in 4m21s
CI / mutation (push) Successful in 7m0s
CI / verify-stack (push) Failing after 30m21s
## What & why

S-18b, second of the S-18 (#19) split (after S-18a #139/#142). Stands up the upstream Maykin **Objecten API** in the compose stack and wires it to the Objecttypen API — the authoritative object store the ACL will write register records to (S-19).

Closes #140

### What

- **Compose** (main + local): `objecten-db` (**PostGIS** — objects carry geometry), `objecten-redis`, `objecten-init` (RUN_SETUP_CONFIG → migrate + provision token + register the Objecttypen service), `objecten` web (health on `/admin/`, host `:8021`). Verbatim upstream image `maykinmedia/objects-api` pinned to `3.4.0` (nearest release to objecttypes-api `3.4.2`; the two speak over the stable Objecttypes API v2).
- **Seed**: `infra/seed-config.sh objecten` streams `infra/objecten/setup_configuration/data.yaml` into the external `rr-objecten-config` volume — same pattern as S-18a. The data.yaml (1) registers **Objecttypen** as a trusted `zgw_consumers` service (`api_type: orc`, api-key auth with the S-18a dev token) so an object can reference its objecttype, and (2) provisions a dev **static API token** so peers (the ACL, S-19) can write objects.
- **Wiring**: added to `WAIT_SVCS`, `CFG_VOLS`, the `SEED` invocations, `seed-config.sh`, and the CI log-dump. `objecten-init` waits on `objecttypen` being healthy so the service registration is meaningful end to end.
- **Smoke**: `verify-objecten` (`infra/run-objecten-check.sh` + `objecten-check.py`) asserts unauth → 401, token → 200 on `/api/v2/objects`; added as a verify-stack step + a row in the #136 check-summary table.

## Verified locally (end to end, real compose)

Seeded + brought up the real `infra/docker-compose.yml` objecten chain (pulls in objecttypen via `depends_on`): `objecten-init` ran setup_configuration — `token_configuration_success` **and** "Successfully executed step: Configuration to connect with external services" — the web reached healthy, and `make verify-objecten` → **"OK — no-auth 401, token 200"**. Confirmed the registered service via the Objecten django shell:

```
objecttypen | orc | http://objecttypen:8000/api/v2/ | api_key
```

YAML (both compose files + ci.yaml) + shell + python all validated; `docker compose config` clean on both files.

## Definition of Done

- [x] Failing smoke committed first (`test(infra): …`, "no running objecten container"); implementation makes it pass.
- [x] Conventional Commits referencing #140.
- [ ] CI green (verify-stack objecten step).
- [x] `docker compose up` reaches health (objecten healthy on first poll locally).
- [x] Demo note in `docs/demo-script.md`.
- [x] Closed by the merging PR (`closes #140`).

No new ADR: follows the established verbatim-image + seed-config CG-module pattern (S-18a/ADR-0023-era).

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #143
2026-07-27 09:53:01 +00:00
not 23ea91de32 feat(infra): Objecttypen API up in compose with a seeded static token (closes #139) (#142)
CI / build (push) Successful in 1m17s
CI / lint (push) Successful in 1m29s
CI / unit (push) Successful in 1m34s
CI / frontend (push) Successful in 3m24s
CI / mutation (push) Successful in 6m33s
CI / verify-stack (push) Failing after 12m2s
## What & why

S-18a, first of the S-18 (#19) split. Stands up the upstream Maykin **Objecttypen API** in the compose stack — the objecttype catalogue the register record (S-18b/S-18c, S-19) will build on.

Closes #139

### What

- **Compose** (main + local): `objecttypen-db` (Postgres), `objecttypen-redis`, `objecttypen-init` (RUN_SETUP_CONFIG → migrate + provision token), `objecttypen` web (health on `/admin/`, host `:8020`). Verbatim upstream image `maykinmedia/objecttypes-api` pinned to `3.4.2`.
- **Seed**: `infra/seed-config.sh objecttypen` streams `infra/objecttypen/setup_configuration/data.yaml` into the external `rr-objecttypen-config` volume — same pattern as OpenZaak/NRC. The data.yaml provisions a dev **static API token** (`tokenauth` setup_configuration step) so peers (Objecten, ACL) can authenticate.
- **Wiring**: added to `WAIT_SVCS`, `CFG_VOLS`, the `SEED` invocations, and the CI log-dump.
- **Smoke**: `verify-objecttypen` (`infra/run-objecttypen-check.sh` + `objecttypen-check.py`) asserts unauth → 401, token → 200; added as a verify-stack step + a row in the #136 check-summary table.

### Split note

#19 was oversized (two CG modules + config + objecttype) → split (§13) into **S-18a** (this), **S-18b** (#140, Objecten wired to Objecttypen), **S-18c** (#141, RegisterRecord objecttype).

## Verified locally (end to end, real compose)

Seeded + brought up the real `infra/docker-compose.yml` objecttypen chain: `objecttypen-init` ran setup_configuration (`token_configuration_success`), the web reached healthy, and `make verify-objecttypen` → **"OK — no-auth 401, token 200"**. YAML (both compose files + ci.yaml) + shell + python all validated.

## Definition of Done

- [x] Smoke check validates the outcome (live, against the running stack).
- [x] Conventional Commits referencing #139.
- [ ] CI green — see note.
- [x] `docker compose up` reaches health (objecttypen healthy on first poll locally).
- [x] Demo note in `docs/demo-script.md`.

## Note on CI

Additive (a new service + its own smoke step). The fast jobs are unaffected. The **verify-stack** job still can't go green until the pre-existing 1.27/act_runner-2.0.0 bring-up P0 is resolved (fails on plain `main` too) — but the objecttypen bring-up itself is validated locally above. No new ADR: this follows the established verbatim-image + seed-config CG-module pattern (ADR-0023-era).

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #142
2026-07-27 08:51:14 +00:00
126 changed files with 6786 additions and 513 deletions
+37 -1
View File
@@ -41,6 +41,27 @@ jobs:
nuget-${{ runner.os }}-
- run: make lint
# The Helm chart's only automated gate: it renders and schema-checks the whole
# stack, and checks it still describes the same stack as the compose file
# (ADR-0033). No cluster involved — see docs/runbooks/kubernetes-talos.md.
k8s:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
# helm as its pinned static binary rather than a marketplace action: one URL,
# the same one the Talos runbook §0 gives a developer, and no third-party
# action to vet (CLAUDE.md §13). The drift check also needs `docker compose`,
# which the runner already has (see docs/runbooks/ci.md).
- name: Install helm
run: |
mkdir -p "$HOME/.local/bin"
curl -sSL https://get.helm.sh/helm-v3.16.4-linux-amd64.tar.gz \
| tar xz -O linux-amd64/helm > "$HOME/.local/bin/helm"
chmod +x "$HOME/.local/bin/helm"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- run: make k8s-lint
- run: make k8s-drift
build:
runs-on: ubuntu-latest
steps:
@@ -204,6 +225,12 @@ jobs:
- name: Objecttypen API up + token authenticates
id: objecttypen
run: OBJECTTYPEN_TIMEOUT=120 make verify-objecttypen
- name: Objecten API up + token authenticates + trusts Objecttypen
id: objecten
run: OBJECTEN_TIMEOUT=120 make verify-objecten
- name: RegisterRecord objecttype registered + published
id: registerrecord
run: REGISTERRECORD_TIMEOUT=120 make verify-registerrecord
- name: ACL ↔ OpenZaak integration tests
id: acl
run: make verify-acl
@@ -213,6 +240,9 @@ jobs:
- name: OpenZaak → NRC → Event Subscriber → projection-api
id: projection
run: make verify-projection
- name: Objecten → NRC notification delivery
id: objecten_nrc
run: make verify-objecten-notifications
- name: Domain → Flowable → ACL → OpenZaak
id: domain
run: make verify-domain
@@ -237,6 +267,9 @@ jobs:
UP: ${{ steps.up.outcome }}
OBS: ${{ steps.obs.outcome }}
OBJECTTYPEN: ${{ steps.objecttypen.outcome }}
OBJECTEN: ${{ steps.objecten.outcome }}
REGISTERRECORD: ${{ steps.registerrecord.outcome }}
OBJECTEN_NOTIFICATIONS: ${{ steps.objecten_nrc.outcome }}
ACL: ${{ steps.acl.outcome }}
NRC: ${{ steps.nrc.outcome }}
PROJECTION: ${{ steps.projection.outcome }}
@@ -256,6 +289,9 @@ jobs:
echo "| Bring up + health | $(icon "$UP") |"
echo "| Observability backplane | $(icon "$OBS") |"
echo "| Objecttypen API + token | $(icon "$OBJECTTYPEN") |"
echo "| Objecten API + token | $(icon "$OBJECTEN") |"
echo "| RegisterRecord objecttype | $(icon "$REGISTERRECORD") |"
echo "| Objecten → NRC | $(icon "$OBJECTEN_NOTIFICATIONS") |"
echo "| ACL ↔ OpenZaak | $(icon "$ACL") |"
echo "| OpenZaak → NRC | $(icon "$NRC") |"
echo "| NRC → Event Subscriber → projection | $(icon "$PROJECTION") |"
@@ -275,7 +311,7 @@ jobs:
# Log dump must precede teardown (which removes the containers).
- name: Dump container logs on failure
if: failure()
run: docker compose -f infra/docker-compose.yml logs --no-color --tail=100 oz-init openzaak nrc-init nrc-web nrc-celery nrc-beat flowable-db flowable-rest flowable-init keycloak acl bff domain projection-db event-subscriber projection-api self-service openbaar behandel beheer objecttypen-db objecttypen-redis objecttypen-init objecttypen tempo prometheus grafana 2>&1 || true
run: docker compose -f infra/docker-compose.yml logs --no-color --tail=100 oz-init openzaak nrc-init nrc-web nrc-celery nrc-beat flowable-db flowable-rest flowable-init keycloak acl bff domain projection-db event-subscriber projection-api self-service openbaar behandel beheer objecttypen-db objecttypen-redis objecttypen-init objecttypen objecten-db objecten-redis objecten-init objecten objecten-celery registerrecord-init tempo prometheus grafana 2>&1 || true
- name: Tear down
if: always()
run: make down
+110
View File
@@ -0,0 +1,110 @@
name: Deploy to Talos
# A merge to main ships the stack to the Talos cluster on the lab server
# (docs/runbooks/kubernetes-talos.md §9). PR CI is the merge gate, so main is
# green by construction — this workflow only deploys.
on:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: read
# Queue deploys, never cancel one: a helm upgrade killed half-way leaves the
# release in `pending-upgrade` and the next run has to be unwedged by hand.
concurrency:
group: deploy-talos
cancel-in-progress: false
jobs:
deploy:
runs-on: ubuntu-latest
env:
# The Talos VM as seen from the Fedora host (libvirt guest IP), and the
# address a browser uses to reach the cluster. `localhost` is deliberate:
# the portals' PKCE needs a secure context, so they are reached over
# `kubectl port-forward` — runbook §5. Override with repo variables.
TALOS_VM_IP: ${{ vars.TALOS_VM_IP }}
TALOS_HOST: ${{ vars.TALOS_HOST }}
steps:
- uses: https://github.com/actions/checkout@v4
# Pinned static binaries, the same URLs the Talos runbook §0 gives a
# developer and the same helm the `k8s` CI job uses — no action to vet.
- name: Install kubectl, helm and crane
run: |
set -euo pipefail
bin="$HOME/.local/bin"; mkdir -p "$bin"
curl -sSLo "$bin/kubectl" https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl
curl -sSL https://get.helm.sh/helm-v3.16.4-linux-amd64.tar.gz | tar xz -O linux-amd64/helm > "$bin/helm"
curl -sSL https://github.com/google/go-containerregistry/releases/download/v0.20.2/go-containerregistry_Linux_x86_64.tar.gz | tar xz -O crane > "$bin/crane"
chmod +x "$bin"/{kubectl,helm,crane}
echo "$bin" >> "$GITHUB_PATH"
# The cluster's API and its registry are only reachable through the Fedora
# host, so forward both to the runner. 30141 is the openbaar portal, for
# the smoke at the end.
- name: Tunnel the Talos API + registry through the Fedora host
env:
SSH_KEY: ${{ secrets.TALOS_SSH_KEY }}
run: |
set -euo pipefail
: "${TALOS_VM_IP:=192.168.122.173}"
umask 077
printf '%s\n' "$SSH_KEY" > ~/.ssh_talos
ssh -i ~/.ssh_talos -o StrictHostKeyChecking=no -o IdentitiesOnly=yes \
-o ExitOnForwardFailure=yes -p 6667 -f -N \
-L 6443:$TALOS_VM_IP:6443 \
-L 30500:$TALOS_VM_IP:30500 \
-L 30141:$TALOS_VM_IP:30141 \
user@labs.respellion.tech
# The kubeconfig's server must be https://127.0.0.1:6443 — Talos puts
# 127.0.0.1 in the apiserver cert SANs, so TLS verification still holds
# through the tunnel.
- name: Write the kubeconfig
env:
KUBECONFIG_B64: ${{ secrets.TALOS_KUBECONFIG }}
run: |
set -euo pipefail
umask 077
base64 -d <<< "$KUBECONFIG_B64" > "$RUNNER_TEMP/kubeconfig"
echo "KUBECONFIG=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_ENV"
kubectl --kubeconfig "$RUNNER_TEMP/kubeconfig" get nodes
# Idempotent; also makes a first deploy onto a bare cluster work. The
# registry's storage is an emptyDir, so a replaced pod loses the images —
# which the push in the next step puts back anyway.
- name: Ensure the in-cluster registry
run: make k8s-registry
# Push through the tunnel (localhost), pull from the node's own NodePort
# (the address in the Talos registry-mirror patch) — same registry, two
# names, so the two `make` calls get different K8S_REGISTRY values.
- name: Build and push the images
run: make k8s-images K8S_REGISTRY=localhost:30500
# k8s-reseed = seed configmaps + helm upgrade + re-run the bootstrap jobs.
# The jobs are idempotent, and deleting them first is what keeps a changed
# Job template from wedging the upgrade (`cannot patch … with kind Job`).
- name: Deploy the chart
run: make k8s-reseed TALOS_HOST=${TALOS_HOST:-localhost} K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500
# `dev` is a mutable tag and helm sees an unchanged pod template, so the
# new images only land on a restart (pullPolicy is already Always).
- name: Roll the services onto the new images
run: |
set -euo pipefail
svcs="acl domain bff event-subscriber projection-api self-service openbaar behandel beheer"
kubectl -n big rollout restart deploy $svcs
kubectl -n big rollout status --timeout=300s deploy $svcs
# Proves portal → Caddy → BFF → projection end to end. An empty register is
# a pass; a 502 or a timeout is not.
- name: Smoke the public register
run: curl -fsS --retry 10 --retry-delay 6 --retry-all-errors http://localhost:30141/openbaar/register
- name: Pods on failure
if: failure()
run: kubectl -n big get pods,jobs || true
+11 -4
View File
@@ -283,16 +283,23 @@ Split into independently deployable sub-slices (CLAUDE.md §13):
Split into independently deployable sub-slices (CLAUDE.md §13):
- **S-18a** (#139) · Objecttypen API up in compose (own DB + seeded config + health + static token).
- **S-18b** (#140) · Objecten API up in compose, wired to Objecttypen. Depends on S-18a.
- **S-18c** (#141) · RegisterRecord objecttype defined + registered (public-safe JSON schema). Depends on S-18a/b.
- **S-18a** (#139, ✅) · Objecttypen API up in compose (own DB + seeded config + health + static token).
- **S-18b** (#140, ✅) · Objecten API up in compose, wired to Objecttypen. Depends on S-18a.
- **S-18c** (#141, ✅) · RegisterRecord objecttype defined + registered (public-safe JSON schema). Depends on S-18a/b.
### S-19 · ACL extension: write register-record to Objecten on approval
### S-19 · ACL extension: write register-record to Objecten on approval *(split — #20 closed)*
**Outcome:** Approval path writes the canonical register record to Objecten, not OpenZaak eigenschappen. Projection now sourced from Objecten events.
**ADR required:** "Why Objecten holds the register, OpenZaak holds the process."
Split into independently deployable sub-slices (CLAUDE.md §13):
- **S-19a** (#149, ✅) · ACL writes the `RegisterRecord` to Objecten on approval, idempotently, alongside the ZGW eindstatus. Carries the ADR (ADR-0028).
- **S-19b** (#150, ✅) · Read projection sourced from Objecten instead of NRC zaak events. *(split — #150 closed)*
- **S-19b-1** (#152, ✅) · Objecten publishes to NRC — broker, celery worker, `objecten` kanaal, notifications config. Turns back on what ADR-0028 deliberately disabled.
- **S-19b-2** (#153, ✅) · Projection derived from `RegisterRecord` objects, rebuildable from the Objecten-derived log. The ACL also writes an INGEDIEND record on submit, so the register holds the whole lifecycle. Carries ADR-0030.
---
## Iteration 5 — Data governance module *(milestone: `Iteration 5 — Data Governance`)*
+122 -7
View File
@@ -10,7 +10,7 @@ COMPOSE := infra/docker-compose.yml
# Long-running services with a healthcheck — the smoke polls these for readiness
# (infra/wait-healthy.sh). One-shot init jobs (oz-init, nrc-init, flowable-init)
# are not polled; they only need to have run. See docs/runbooks/gitea-actions-gotchas.md.
WAIT_SVCS := openzaak nrc-web acl bff domain event-subscriber projection-api self-service openbaar behandel beheer objecttypen
WAIT_SVCS := openzaak nrc-web acl bff domain event-subscriber projection-api self-service openbaar behandel beheer objecttypen objecten
# Config files (OpenZaak data.yaml, Keycloak realms, Flowable BPMN) are streamed
# into external named volumes via `docker cp` (infra/seed-config.sh) instead of
# bind-mounted, because bind mounts don't reach sibling containers on the
@@ -18,7 +18,7 @@ WAIT_SVCS := openzaak nrc-web acl bff domain event-subscriber projection-api se
# volumes are `external`, so compose won't remove them — CFG_VOLS lists them for
# explicit teardown. See docs/runbooks/gitea-actions-gotchas.md.
SEED := bash infra/seed-config.sh
CFG_VOLS := rr-oz-config rr-nrc-config rr-kc-realms rr-fl-bpmn rr-objecttypen-config
CFG_VOLS := rr-oz-config rr-nrc-config rr-kc-realms rr-fl-bpmn rr-objecttypen-config rr-objecten-config rr-registerrecord-config
# Local-only stack: same services but config is bind-mounted (no seed step), so a
# plain `docker compose -f infra/docker-compose.local.yml up` works on any local
# engine. This is the no-make / Windows-friendly path. See that file's header.
@@ -43,7 +43,7 @@ export DOCKER_HOST := unix://$(PODMAN_SOCK)
endif
endif
.PHONY: ci lint build unit mutation frontend integration verify verify-up verify-acl verify-nrc verify-projection verify-bff verify-domain verify-observability verify-tracing verify-metrics verify-objecttypen verify-notifications smoke up down local verify-local local-down changelog openzaak-up openzaak-smoke openzaak-seed openzaak-down stack-up stack-smoke stack-down keycloak-up keycloak-smoke keycloak-down flowable-up flowable-smoke flowable-down help
.PHONY: ci lint build unit mutation frontend integration verify verify-up verify-acl verify-nrc verify-projection verify-bff verify-domain verify-observability verify-tracing verify-metrics verify-objecttypen verify-objecten verify-registerrecord verify-objecten-notifications verify-notifications smoke up down local verify-local local-down changelog openzaak-up openzaak-smoke openzaak-seed openzaak-down stack-up stack-smoke stack-down keycloak-up keycloak-smoke keycloak-down flowable-up flowable-smoke flowable-down k8s-lint k8s-drift k8s-registry k8s-images k8s-seed k8s-up k8s-reseed k8s-portals k8s-down k8s-purge help
## ci: run the full pipeline — lint, build, unit, mutation, frontend, verify (mirrors Gitea Actions)
## `verify` is the live-stack stage (full stack up once → ACL + notification checks).
@@ -64,6 +64,9 @@ frontend:
## lint: verify formatting (no changes)
lint:
dotnet format $(SLN) --verify-no-changes
# Only pages in mkdocs.yml's nav are published, and mkdocs keeps a build green
# when one is missing — so the nav is checked here rather than not at all.
python3 infra/check-docs-nav.py
## build: release build
build:
@@ -71,8 +74,12 @@ build:
## unit: run unit tests (excludes the container-backed Integration lane)
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
# The CI reporting scripts are stdlib Python with their own assert-based self-checks (#161) — they
# ride this lane so a broken job summary is caught by CI rather than by the next red pipeline.
unit:
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
python3 infra/test_playwright_summary.py
python3 infra/test_portal_caddyfiles.py
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
# Stryker is pinned as a local dotnet tool (.config/dotnet-tools.json); `tool restore`
@@ -94,14 +101,14 @@ mutation:
# podman-compose, and needing no `--wait` flag or host port access. The one-shots
# (oz-init, flowable-init) aren't polled; they just need to have run.
smoke:
$(SEED) oz nrc kc fl objecttypen
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
docker compose -f $(COMPOSE) up -d --build
bash -c 'WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS); rc=$$?; docker compose -f $(COMPOSE) down --volumes; docker volume rm -f $(CFG_VOLS) >/dev/null 2>&1; exit $$rc'
## up: seed config volumes and start the full stack (use instead of bare
## `docker compose up`, which can't self-seed the external config volumes)
up:
$(SEED) oz nrc kc fl objecttypen
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
docker compose -f $(COMPOSE) up -d --build
## down: stop and remove the local stack (incl. the external config volumes)
@@ -139,7 +146,7 @@ changelog:
## verify-up: bring the FULL stack up and wait for health (CI verify-stack step 1;
## subsumes the old compose-smoke health gate — the DoD "up reaches green" check).
verify-up:
$(SEED) oz nrc kc fl objecttypen
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
docker compose -f $(COMPOSE) up -d --build
WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS)
@@ -191,17 +198,33 @@ verify-metrics:
verify-objecttypen:
bash infra/run-objecttypen-check.sh
## verify-objecten: assert the Objecten API is up + its static token authenticates and it
## trusts the Objecttypen API (S-18b), against the already-running stack.
verify-objecten:
bash infra/run-objecten-check.sh
## verify-registerrecord: assert the RegisterRecord objecttype is registered + published in the
## Objecttypen API (S-18c), against the already-running stack.
verify-registerrecord:
bash infra/run-registerrecord-check.sh
## verify-objecten-notifications: assert a RegisterRecord write in Objecten is DELIVERED as an
## `objecten` notification via NRC (S-19b-1), against the already-running stack.
verify-objecten-notifications:
bash infra/run-objecten-notifications-check.sh
## verify: local mirror of the CI verify-stack job — full stack up once, all checks,
## tear down (always). For fast single-concern local iteration use `integration`
## (oz-only) or `verify-notifications` (oz+nrc) instead.
verify:
$(SEED) oz nrc kc fl objecttypen
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
docker compose -f $(COMPOSE) up -d --build
@bash -c 'set -e; rc=0; \
WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS) \
&& bash infra/run-acl-integration.sh \
&& bash infra/run-notification-check.sh \
&& bash infra/run-projection-check.sh \
&& bash infra/run-objecten-notifications-check.sh \
&& bash infra/run-domain-check.sh \
&& bash infra/run-bff-check.sh \
&& bash infra/run-e2e-check.sh || rc=$$?; \
@@ -310,6 +333,98 @@ flowable-down:
docker compose -f $(FL_COMPOSE) down --volumes
-docker volume rm -f rr-fl-bpmn
# ── Kubernetes (single-node Talos) ─────────────────────────────────────────────
# The Helm chart in infra/helm/big-reference is a port of infra/docker-compose.yml
# (ADR-0033). Full walkthrough: docs/runbooks/kubernetes-talos.md.
# TALOS_HOST the address the BROWSER uses — pins Keycloak's issuer and the portals'
# OIDC authority. Use `localhost` with `make k8s-portals`: the OIDC
# library needs crypto.subtle, which browsers only expose on a secure
# context (https, or localhost) — see docs/runbooks/kubernetes-talos.md §5
# K8S_REGISTRY the registry both sides use for this repo's images (see k8s-registry)
K8S_NS ?= big
K8S_CHART := infra/helm/big-reference
K8S_REGISTRY ?=
TALOS_HOST ?=
# The images built from this repo — compose service name == image name == chart workload.
K8S_IMAGES := acl domain bff event-subscriber projection-api self-service openbaar behandel beheer
## k8s-lint: render + schema-check the Helm chart (no cluster needed)
k8s-lint:
helm lint $(K8S_CHART)
helm template big $(K8S_CHART) -n $(K8S_NS) --set images.registry=registry.invalid:5000 >/dev/null
## k8s-drift: fail if compose and the Helm chart describe different stacks
# Compose is CI-canonical (ADR-0033) and the chart is a transcription of it; this
# compares what each one deploys — workload names and resolved images. Needs
# `docker compose` and `helm`, no cluster.
k8s-drift:
python3 infra/helm/check-drift.py
## k8s-registry: deploy the in-cluster image registry (NodePort 30500)
k8s-registry:
kubectl apply -f infra/helm/registry.yaml
kubectl -n registry rollout status deploy/registry --timeout=180s
## k8s-images: build this repo's images (via compose) and push them to $(K8S_REGISTRY)
# `docker save | crane push` rather than `docker push`: the registry speaks plain
# HTTP, which the Docker daemon refuses without a root-level insecure-registries
# entry, while crane just takes --insecure. Install: see docs/runbooks/kubernetes-talos.md.
k8s-images:
@command -v crane >/dev/null || { echo "crane not found — see docs/runbooks/kubernetes-talos.md §0" >&2; exit 2; }
@test -n "$(K8S_REGISTRY)" || { echo "set K8S_REGISTRY=<registry host:port>" >&2; exit 2; }
docker compose -f $(COMPOSE) build $(K8S_IMAGES)
@tar=$$(mktemp -t rr-img-XXXX.tar); \
for i in $(K8S_IMAGES); do \
docker save register-referentie/$$i:dev -o $$tar; \
crane push --insecure $$tar $(K8S_REGISTRY)/register-referentie/$$i:dev; \
done; rm -f $$tar
## k8s-seed: create the ConfigMaps the chart mounts (upstream config + bootstrap scripts)
k8s-seed:
bash infra/helm/seed-configmaps.sh $(K8S_NS)
## k8s-up: seed the config and install/upgrade the release
k8s-up: k8s-seed
@test -n "$(TALOS_HOST)" || { echo "set TALOS_HOST=<node ip>" >&2; exit 2; }
@test -n "$(K8S_REGISTRY)" || { echo "set K8S_REGISTRY=<registry the node can pull from>" >&2; exit 2; }
helm upgrade --install big $(K8S_CHART) -n $(K8S_NS) --create-namespace \
--set host=$(TALOS_HOST) --set images.registry=$(K8S_REGISTRY) $(K8S_SET)
kubectl -n $(K8S_NS) get pods
## k8s-reseed: re-run the bootstrap jobs (after a database was wiped, or after
## changing a Job in the chart — Job pod templates are immutable, so a plain
## `helm upgrade` is rejected)
k8s-reseed:
kubectl -n $(K8S_NS) delete job -l app.kubernetes.io/component=init --ignore-not-found
$(MAKE) k8s-up
# The projection's schema is created on service start (Projection.ReadModel migrates in a
# hosted service), so a wiped database also needs these two restarted — otherwise they keep
# writing to a schema-less DB and fail with `relation "processed_notifications" does not exist`.
kubectl -n $(K8S_NS) rollout restart deploy/event-subscriber deploy/projection-api
kubectl -n $(K8S_NS) rollout status deploy/event-subscriber deploy/projection-api --timeout=180s
## k8s-portals: forward the browser-facing services to localhost (Ctrl-C stops them all)
# The portals' OIDC flow needs a *secure context* for crypto.subtle (PKCE), and browsers
# only grant that to https or localhost — a NodePort on the VM's IP is neither. Forwarding
# to localhost on the same port numbers keeps Keycloak's pinned issuer valid. Deploy with
# TALOS_HOST=localhost for this to line up.
k8s-portals:
@echo "self-service http://localhost:30140 · openbaar :30141 · behandel :30142 · beheer :30143 · keycloak :30180"
@trap 'kill 0' INT TERM; \
for f in self-service:30140:80 openbaar:30141:80 behandel:30142:80 beheer:30143:80 keycloak:30180:8080; do \
svc=$${f%%:*}; rest=$${f#*:}; lport=$${rest%%:*}; rport=$${rest#*:}; \
kubectl -n $(K8S_NS) port-forward --address 127.0.0.1 svc/$$svc $$lport:$$rport >/dev/null & \
done; wait
## k8s-down: uninstall the release (database PVCs are kept)
k8s-down:
helm uninstall big -n $(K8S_NS)
## k8s-purge: uninstall AND drop the namespace, including the database volumes
k8s-purge:
-helm uninstall big -n $(K8S_NS)
kubectl delete namespace $(K8S_NS) --ignore-not-found
## help: list available targets
help:
@grep -E '^## ' $(MAKEFILE_LIST) | sed 's/^## //'
+22
View File
@@ -0,0 +1,22 @@
:80 {
# Same-origin API: behandelaars authenticate against the medewerker realm; the BFF validates it
# for /behandel/* (S-12c).
# `handle` blocks are mutually exclusive and matched most-specific-first, so the
# SPA fallback below can never swallow an API call — unlike a bare `try_files`,
# which Caddy sorts *before* reverse_proxy and would rewrite it to /index.html.
#
# No `resolver` stanza is needed: Caddy dials the upstream per
# request through the system resolver, so it starts before the BFF is up, picks up
# its restarts, and honours the DNS search domains in /etc/resolv.conf — which is
# what lets the bare `bff` name resolve on Kubernetes as well as under compose.
handle /behandel/* {
reverse_proxy bff:8080
}
# The Angular app. Client-side routing: an unknown path serves index.html.
handle {
root * /usr/share/caddy
try_files {path} /index.html
file_server
}
}
+6 -9
View File
@@ -1,4 +1,4 @@
# Multi-stage build for the behandel portal (Angular → nginx).
# Multi-stage build for the behandel portal (Angular → Caddy).
# Build context is the repo root (the app needs the pnpm workspace + libs). See infra/docker-compose.yml.
FROM node:24-slim AS build
WORKDIR /src
@@ -13,15 +13,12 @@ COPY apps/behandel apps/behandel
COPY libs libs
RUN pnpm nx build behandel
FROM nginx:1.27-alpine AS runtime
COPY apps/behandel/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/dist/apps/behandel/browser /usr/share/nginx/html
FROM caddy:2-alpine AS runtime
COPY apps/behandel/Caddyfile /etc/caddy/Caddyfile
COPY --from=build /src/dist/apps/behandel/browser /usr/share/caddy
# Compose-time OIDC config: the browser (Playwright, on the compose network) reaches Keycloak by
# service name, so the token issuer matches the BFF's medewerker authority (host-consistent, ADR-0013).
RUN printf '{ "authority": "http://keycloak:8080/realms/medewerker" }\n' > /usr/share/nginx/html/config.json
# Make the reverse-proxy resolver engine-portable (Docker 127.0.0.11 vs podman aardvark); runs from
# the nginx image's /docker-entrypoint.d before nginx starts.
COPY apps/portal-nginx-resolver.sh /docker-entrypoint.d/40-resolver.sh
RUN chmod +x /docker-entrypoint.d/40-resolver.sh
# Kubernetes mounts a ConfigMap over this file with the node address instead (ADR-0033).
RUN printf '{ "authority": "http://keycloak:8080/realms/medewerker" }\n' > /usr/share/caddy/config.json
EXPOSE 80
-24
View File
@@ -1,24 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Resolve the BFF via Docker's embedded DNS at request time (variable proxy_pass), so nginx starts
# even before the BFF is up and picks up restarts — instead of failing to load the config.
resolver 127.0.0.11 ipv6=off valid=30s;
# Same-origin API: proxy the behandel endpoint group to the bff service. The api-client uses
# relative URLs, so the browser calls this origin and nginx forwards to the BFF — no CORS, and the
# medewerker token (same-origin) is attached by the app's interceptor (ADR-0013).
location /behandel/ {
set $bff http://bff:8080;
proxy_pass $bff;
proxy_set_header Host $host;
}
# SPA fallback — Angular client-side routing.
location / {
try_files $uri $uri/ /index.html;
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ export interface RuntimeConfig {
/**
* Route prefixes whose requests carry the medewerker token. These MUST match the **relative** URLs
* the api-client actually calls (same-origin via the nginx proxy) — the interceptor matches on
* the api-client actually calls (same-origin via the Caddy proxy) — the interceptor matches on
* `req.url`, which stays relative, so an absolute origin would never match and the token would go
* unattached. Only `/behandel/` is secured; the app calls no other endpoint group.
*/
@@ -4,7 +4,7 @@ import { of, throwError } from 'rxjs';
import { BffApiV1Service, type WerkbakItem } from 'api-client';
import { AuthService } from 'auth';
import { axe } from 'vitest-axe';
import { WerkbakPage } from './werkbak-page';
import { WERKBAK_REFRESH_MS, WerkbakPage } from './werkbak-page';
const sample: WerkbakItem[] = [
{ registrationId: 'reg-1', bsn: '123456782', status: 'InBehandeling' },
@@ -81,6 +81,94 @@ describe('WerkbakPage', () => {
});
});
it('picks up a newly submitted registration without a reload', async () => {
// S-26 (#162): a registration reaches Beoordelen asynchronously, after the citizen supplies
// documents — so the werkbak must refresh itself rather than wait for the behandelaar to reload.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(of([sample[0]]))
.mockReturnValue(of(sample));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
expect(screen.getByText('reg-1')).toBeTruthy();
expect(screen.queryByText('reg-2')).toBeNull();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(getBehandelWerkbak).toHaveBeenCalledTimes(2);
expect(screen.getByText('reg-2')).toBeTruthy();
// A background refresh must not flash the loading state over the rows the behandelaar is reading.
expect(screen.queryByText(/bezig met laden/i)).toBeNull();
} finally {
vi.useRealTimers();
}
});
it('keeps the rows on screen when a background refresh fails', async () => {
// A blip on a background poll must not replace the list with the load-failure alert; the next
// tick recovers. Only the first load speaks for whether the werkbak is readable at all.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(of(sample))
.mockReturnValue(throwError(() => new Error('503')));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(screen.getByText('reg-1')).toBeTruthy();
expect(screen.queryByText(/kon de werkbak niet laden/i)).toBeNull();
} finally {
vi.useRealTimers();
}
});
it('stops refreshing once the page is destroyed', async () => {
vi.useFakeTimers();
try {
const { getBehandelWerkbak, providers } = setup();
const { fixture } = await render(WerkbakPage, { providers });
fixture.destroy();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS * 3);
expect(getBehandelWerkbak).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
}
});
it('clears a load failure once a refresh succeeds', async () => {
// Without this the werkbak stays stuck on the error until the behandelaar reloads — the very
// thing this slice removes. A recovered read must put the rows back.
vi.useFakeTimers();
try {
const getBehandelWerkbak = vi
.fn()
.mockReturnValueOnce(throwError(() => new Error('503')))
.mockReturnValue(of(sample));
const { providers } = setup({ getBehandelWerkbak });
const { detectChanges } = await render(WerkbakPage, { providers });
expect(screen.getByText(/kon de werkbak niet laden/i)).toBeTruthy();
vi.advanceTimersByTime(WERKBAK_REFRESH_MS);
detectChanges();
expect(screen.queryByText(/kon de werkbak niet laden/i)).toBeNull();
expect(screen.getByText('reg-1')).toBeTruthy();
} finally {
vi.useRealTimers();
}
});
it('shows an empty state when the werkbak has no items', async () => {
const { providers } = setup({ getBehandelWerkbak: vi.fn().mockReturnValue(of([])) });
await render(WerkbakPage, { providers });
+34 -3
View File
@@ -1,7 +1,15 @@
import { Component, inject, signal } from '@angular/core';
import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
import { interval } from 'rxjs';
import { BffApiV1Service, type WerkbakItem } from 'api-client';
import { UtrechtComponentsModule } from 'ui';
/**
* How often an open werkbak re-reads itself (S-26/#162, ADR-0032). Exported so the spec advances the
* clock by exactly one interval instead of hard-coding the number.
*/
export const WERKBAK_REFRESH_MS = 5_000;
/** The two decisions a behandelaar can make; the BFF validates these exact values (ADR-0013). */
type Besluit = 'goedkeuren' | 'afwijzen';
@@ -10,6 +18,11 @@ type Besluit = 'goedkeuren' | 'afwijzen';
* Flowable `Beoordelen` tasks, read through the domain) and decides each — goedkeuren or afwijzen. A
* decision posts to the BFF, which applies the domain transition and completes the workflow task
* (ADR-0013; S-12). After a decision the werkbak refreshes so the handled item drops off the list.
*
* The page also re-reads itself every {@link WERKBAK_REFRESH_MS} while it is open, so a registration
* that reaches beoordeling after the behandelaar opened the werkbak shows up on its own — no reload
* (S-26/#162). Polling rather than a pushed stream: nothing notifies the BFF either, so a stream
* would poll the domain in the BFF instead and add connection state for the same freshness (ADR-0032).
*/
@Component({
selector: 'app-werkbak-page',
@@ -27,19 +40,37 @@ export class WerkbakPage {
constructor() {
this.load();
// ponytail: a fixed interval, polled while the page lives — it keeps refreshing in a background
// tab. Gate on `document.visibilityState` if the request volume ever matters.
interval(WERKBAK_REFRESH_MS)
.pipe(takeUntilDestroyed())
.subscribe(() => this.load({ background: true }));
}
load(): void {
this.loading.set(true);
this.failed.set(false);
/**
* Read the werkbak. A `background` read is the interval refresh: it leaves the rows and the states
* the behandelaar is looking at alone until it has an answer — no loading flash on every tick, and
* a blip does not swap the list for the failure alert (the next tick recovers). Only a foreground
* read — on open, or after a decision — speaks for whether the werkbak is readable at all.
*/
load(options: { background?: boolean } = {}): void {
const background = options.background ?? false;
if (!background) {
this.loading.set(true);
this.failed.set(false);
}
this.bff.getBehandelWerkbak().subscribe({
next: (rows: WerkbakItem[]) => {
this.items.set(rows);
this.loading.set(false);
this.loaded.set(true);
// A read that came back is the answer, so a refresh also clears an earlier failure — the
// werkbak recovers on its own instead of showing the error until someone reloads.
this.failed.set(false);
},
// Surface the failure (e.g. 403 for a non-behandelaar) instead of swallowing it.
error: () => {
if (background) return;
this.items.set([]);
this.loading.set(false);
this.loaded.set(true);
+21
View File
@@ -0,0 +1,21 @@
:80 {
# Same-origin API: beheerders use the same medewerker realm as behandel (S-15a).
# `handle` blocks are mutually exclusive and matched most-specific-first, so the
# SPA fallback below can never swallow an API call — unlike a bare `try_files`,
# which Caddy sorts *before* reverse_proxy and would rewrite it to /index.html.
#
# No `resolver` stanza is needed: Caddy dials the upstream per
# request through the system resolver, so it starts before the BFF is up, picks up
# its restarts, and honours the DNS search domains in /etc/resolv.conf — which is
# what lets the bare `bff` name resolve on Kubernetes as well as under compose.
handle /beheer/* {
reverse_proxy bff:8080
}
# The Angular app. Client-side routing: an unknown path serves index.html.
handle {
root * /usr/share/caddy
try_files {path} /index.html
file_server
}
}
+6 -9
View File
@@ -1,4 +1,4 @@
# Multi-stage build for the beheer portal (Angular → nginx).
# Multi-stage build for the beheer portal (Angular → Caddy).
# Build context is the repo root (the app needs the pnpm workspace + libs). See infra/docker-compose.yml.
FROM node:24-slim AS build
WORKDIR /src
@@ -13,15 +13,12 @@ COPY apps/beheer apps/beheer
COPY libs libs
RUN pnpm nx build beheer
FROM nginx:1.27-alpine AS runtime
COPY apps/beheer/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/dist/apps/beheer/browser /usr/share/nginx/html
FROM caddy:2-alpine AS runtime
COPY apps/beheer/Caddyfile /etc/caddy/Caddyfile
COPY --from=build /src/dist/apps/beheer/browser /usr/share/caddy
# Compose-time OIDC config: the browser (Playwright, on the compose network) reaches Keycloak by
# service name, so the token issuer matches the BFF's medewerker authority (host-consistent, ADR-0013).
RUN printf '{ "authority": "http://keycloak:8080/realms/medewerker" }\n' > /usr/share/nginx/html/config.json
# Make the reverse-proxy resolver engine-portable (Docker 127.0.0.11 vs podman aardvark); runs from
# the nginx image's /docker-entrypoint.d before nginx starts.
COPY apps/portal-nginx-resolver.sh /docker-entrypoint.d/40-resolver.sh
RUN chmod +x /docker-entrypoint.d/40-resolver.sh
# Kubernetes mounts a ConfigMap over this file with the node address instead (ADR-0033).
RUN printf '{ "authority": "http://keycloak:8080/realms/medewerker" }\n' > /usr/share/caddy/config.json
EXPOSE 80
-24
View File
@@ -1,24 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Resolve the BFF via Docker's embedded DNS at request time (variable proxy_pass), so nginx starts
# even before the BFF is up and picks up restarts — instead of failing to load the config.
resolver 127.0.0.11 ipv6=off valid=30s;
# Same-origin API: proxy the beheer endpoint group to the bff service. The api-client uses
# relative URLs, so the browser calls this origin and nginx forwards to the BFF — no CORS, and the
# medewerker token (same-origin) is attached by the app's interceptor (ADR-0013).
location /beheer/ {
set $bff http://bff:8080;
proxy_pass $bff;
proxy_set_header Host $host;
}
# SPA fallback — Angular client-side routing.
location / {
try_files $uri $uri/ /index.html;
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ export interface RuntimeConfig {
/**
* Route prefixes whose requests carry the medewerker token. These MUST match the **relative** URLs
* the api-client actually calls (same-origin via the nginx proxy) — the interceptor matches on
* the api-client actually calls (same-origin via the Caddy proxy) — the interceptor matches on
* `req.url`, which stays relative, so an absolute origin would never match and the token would go
* unattached. Only `/beheer/` is secured; the app calls no other endpoint group.
*/
+21
View File
@@ -0,0 +1,21 @@
:80 {
# Same-origin API: the public register is anonymous, but still reads through the BFF (S-09).
# `handle` blocks are mutually exclusive and matched most-specific-first, so the
# SPA fallback below can never swallow an API call — unlike a bare `try_files`,
# which Caddy sorts *before* reverse_proxy and would rewrite it to /index.html.
#
# No `resolver` stanza is needed: Caddy dials the upstream per
# request through the system resolver, so it starts before the BFF is up, picks up
# its restarts, and honours the DNS search domains in /etc/resolv.conf — which is
# what lets the bare `bff` name resolve on Kubernetes as well as under compose.
handle /openbaar/* {
reverse_proxy bff:8080
}
# The Angular app. Client-side routing: an unknown path serves index.html.
handle {
root * /usr/share/caddy
try_files {path} /index.html
file_server
}
}
+4 -8
View File
@@ -1,4 +1,4 @@
# Multi-stage build for the openbaar portal (Angular → nginx).
# Multi-stage build for the openbaar portal (Angular → Caddy).
# Build context is the repo root (the app needs the pnpm workspace + libs). See infra/docker-compose.yml.
FROM node:24-slim AS build
WORKDIR /src
@@ -13,13 +13,9 @@ COPY apps/openbaar apps/openbaar
COPY libs libs
RUN pnpm nx build openbaar
FROM nginx:1.27-alpine AS runtime
COPY apps/openbaar/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/dist/apps/openbaar/browser /usr/share/nginx/html
FROM caddy:2-alpine AS runtime
COPY apps/openbaar/Caddyfile /etc/caddy/Caddyfile
COPY --from=build /src/dist/apps/openbaar/browser /usr/share/caddy
# No runtime config: the openbaar register is anonymous (no OIDC authority to inject).
# Make the reverse-proxy resolver engine-portable (Docker 127.0.0.11 vs podman aardvark); runs from
# the nginx image's /docker-entrypoint.d before nginx starts.
COPY apps/portal-nginx-resolver.sh /docker-entrypoint.d/40-resolver.sh
RUN chmod +x /docker-entrypoint.d/40-resolver.sh
EXPOSE 80
-23
View File
@@ -1,23 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Resolve the BFF via Docker's embedded DNS at request time (variable proxy_pass), so nginx starts
# even before the BFF is up and picks up restarts — instead of failing to load the config.
resolver 127.0.0.11 ipv6=off valid=30s;
# Same-origin API: proxy the anonymous openbaar endpoint group to the bff service. The api-client
# uses relative URLs, so the browser calls this origin and nginx forwards to the BFF — no CORS.
location /openbaar/ {
set $bff http://bff:8080;
proxy_pass $bff;
proxy_set_header Host $host;
}
# SPA fallback — Angular client-side routing.
location / {
try_files $uri $uri/ /index.html;
}
}
+1 -1
View File
@@ -8,7 +8,7 @@ import { appRoutes } from './app.routes';
/**
* The openbaar register is a public, anonymous read: no DigiD, no auth interceptor. The app is served
* same-origin as the BFF (nginx proxies /openbaar), so the api-client's relative calls stay same-origin.
* same-origin as the BFF (Caddy proxies /openbaar), so the api-client's relative calls stay same-origin.
*/
export const appConfig: ApplicationConfig = {
providers: [
-17
View File
@@ -1,17 +0,0 @@
#!/bin/sh
# Point nginx's reverse-proxy `resolver` at THIS container's real DNS server.
#
# The portal nginx configs use a variable proxy_pass, which needs a `resolver` so the BFF hostname is
# resolved at request time (nginx can start before the BFF is up). The config hardcodes Docker's
# embedded DNS (127.0.0.11) — correct on Docker/Docker Desktop, but rootless podman uses a
# network-specific address (aardvark, e.g. 10.89.0.1), so proxied calls 502 there. Read the actual
# nameserver from /etc/resolv.conf and substitute it, so the reverse proxy works on any engine.
#
# Runs from the nginx image's /docker-entrypoint.d/ before nginx starts. On Docker the nameserver IS
# 127.0.0.11, so the substitution is a no-op. Guarded (no `set -e`) so it's safe whether the nginx
# entrypoint executes or sources it.
ns="$(awk '/^nameserver/{print $2; exit}' /etc/resolv.conf 2>/dev/null)"
if [ -n "$ns" ] && [ "$ns" != "127.0.0.11" ]; then
sed -i "s/resolver 127\.0\.0\.11/resolver $ns/" /etc/nginx/conf.d/default.conf 2>/dev/null || true
echo "portal-nginx-resolver: set resolver to $ns"
fi
+26
View File
@@ -0,0 +1,26 @@
:80 {
# Same-origin API: the api-client uses relative URLs, so the browser calls this origin and Caddy
# forwards to the BFF — no CORS, and the DigiD token is attached by the app interceptor
# (S-08d/ADR-0010).
# `handle` blocks are mutually exclusive and matched most-specific-first, so the
# SPA fallback below can never swallow an API call — unlike a bare `try_files`,
# which Caddy sorts *before* reverse_proxy and would rewrite it to /index.html.
#
# No `resolver` stanza is needed: Caddy dials the upstream per
# request through the system resolver, so it starts before the BFF is up, picks up
# its restarts, and honours the DNS search domains in /etc/resolv.conf — which is
# what lets the bare `bff` name resolve on Kubernetes as well as under compose.
handle /self-service/* {
reverse_proxy bff:8080
}
handle /openbaar/* {
reverse_proxy bff:8080
}
# The Angular app. Client-side routing: an unknown path serves index.html.
handle {
root * /usr/share/caddy
try_files {path} /index.html
file_server
}
}
+6 -9
View File
@@ -1,4 +1,4 @@
# Multi-stage build for the self-service portal (Angular → nginx).
# Multi-stage build for the self-service portal (Angular → Caddy).
# Build context is the repo root (the app needs the pnpm workspace + libs). See infra/docker-compose.yml.
FROM node:24-slim AS build
WORKDIR /src
@@ -13,15 +13,12 @@ COPY apps/self-service apps/self-service
COPY libs libs
RUN pnpm nx build self-service
FROM nginx:1.27-alpine AS runtime
COPY apps/self-service/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /src/dist/apps/self-service/browser /usr/share/nginx/html
FROM caddy:2-alpine AS runtime
COPY apps/self-service/Caddyfile /etc/caddy/Caddyfile
COPY --from=build /src/dist/apps/self-service/browser /usr/share/caddy
# Compose-time OIDC config: the browser (Playwright, on the compose network) reaches Keycloak by
# service name, so the token issuer matches the BFF's authority (host-consistent, ADR-0010).
RUN printf '{ "authority": "http://keycloak:8080/realms/digid" }\n' > /usr/share/nginx/html/config.json
# Make the reverse-proxy resolver engine-portable (Docker 127.0.0.11 vs podman aardvark); runs from
# the nginx image's /docker-entrypoint.d before nginx starts.
COPY apps/portal-nginx-resolver.sh /docker-entrypoint.d/40-resolver.sh
RUN chmod +x /docker-entrypoint.d/40-resolver.sh
# Kubernetes mounts a ConfigMap over this file with the node address instead (ADR-0033).
RUN printf '{ "authority": "http://keycloak:8080/realms/digid" }\n' > /usr/share/caddy/config.json
EXPOSE 80
-29
View File
@@ -1,29 +0,0 @@
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Resolve the BFF via Docker's embedded DNS at request time (variable proxy_pass), so nginx starts
# even before the BFF is up and picks up restarts — instead of failing to load the config.
resolver 127.0.0.11 ipv6=off valid=30s;
# Same-origin API: proxy the BFF endpoint groups to the bff service. The api-client uses relative
# URLs, so the browser calls this origin and nginx forwards to the BFF — no CORS, and the DigiD
# token (same-origin) is attached by the app's interceptor (S-08d/ADR-0010).
location /self-service/ {
set $bff http://bff:8080;
proxy_pass $bff;
proxy_set_header Host $host;
}
location /openbaar/ {
set $bff http://bff:8080;
proxy_pass $bff;
proxy_set_header Host $host;
}
# SPA fallback — Angular client-side routing.
location / {
try_files $uri $uri/ /index.html;
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ export interface RuntimeConfig {
/**
* Route prefixes whose requests carry the DigiD token. These MUST match the **relative** URLs the
* api-client actually calls (same-origin via the nginx proxy) — the interceptor matches on `req.url`,
* api-client actually calls (same-origin via the Caddy proxy) — the interceptor matches on `req.url`,
* which stays relative, so an absolute origin would never match and the token would go unattached.
* `/openbaar/` is deliberately excluded: it is the anonymous public register.
*/
+1 -1
View File
@@ -207,7 +207,7 @@ A slice is done when:
## 15. Out of scope for v1
- OpenMetadata data governance module (v3 slice).
- Objecten as the authoritative register record store (v2 slice — v1 uses OpenZaak zaak-eigenschappen as a placeholder).
- ~~Objecten as the authoritative register record store~~ — **delivered** in S-19a (#149, ADR-0028); the approval path writes a `RegisterRecord` object to Objecten rather than the planned zaak-eigenschappen placeholder.
- Production-grade Helm chart (sketch only).
- Multi-tenancy.
- Real outbound notifications (email/SMS) — logged to console in v1.
@@ -67,6 +67,14 @@ itself, so no in-image healthcheck tool is required.
- Three more images built each CI run (kept small; not on the health-gate list).
- Storage is ephemeral container fs — a demo backplane, not a retention target.
Object storage for Tempo / remote-write for Prometheus is a later concern.
- Tempo runs **single-binary**, so its distributor and ingester are one process and
some of its distributed-mode machinery is not just redundant but harmful. Its
ingester-pool health check is disabled (`ingester_client.pool_config`) because with
a single in-process ingester the check can never route around a failure — a 1s
loopback-gRPC deadline missed under CI load only evicted the one ingester and made
Tempo drop spans, which is how `verify-tracing` flaked (#156). Expect the same
shape from other distributed-mode knobs if we tune them; the fix is to switch to
real multi-ingester Tempo, not to re-enable them here.
## Coupling rules touched (CLAUDE.md §8)
@@ -0,0 +1,81 @@
# ADR-0027: The RegisterRecord objecttype is public-safe by construction
- **Status:** Accepted
- **Date:** 2026-07-27
- **Deciders:** Respellion engineering
- **Slice:** S-18c (#141), third of the S-18 (#19) split
## Context
S-18 stands up Objecttypen (S-18a) and Objecten (S-18b) as the authoritative
register-record store (PRD §"Objecten as the authoritative register record store").
S-19 (#20) will, on approval, write the canonical register record to the Objecten API
instead of OpenZaak zaak-eigenschappen, and the openbaar (public) register will read it.
Objecten validates every object against a **objecttype version's JSON schema**. So the
schema is a contract: it fixes which fields a register record may carry. The register is
read **anonymously** by the openbaar portal (ADR-0010), so the schema is also a
disclosure boundary — anything the schema allows can end up public.
Two questions: **which fields** the schema defines, and **how** the objecttype gets into
the Objecttypen API (which has no declarative objecttype step).
## Decision
**Define a `RegisterRecord` objecttype whose published schema carries exactly the
public-safe fields — `id`, `status`, `reference` — and register it over the API at
startup with a one-shot, idempotently.**
### The schema mirrors the BFF's public projection, not the internal one
The public-safe field set already exists: the BFF's `OpenbaarEntry`
(`services/bff/Bff.Api/DownstreamClients.cs`) — `id`, `status`, `reference` — is what
`OpenbaarProjection.PublicView` narrows every row down to, dropping `bsn` and
`naamPlaceholder` at the boundary (S-09). The RegisterRecord schema mirrors that record,
**not** the internal `RegisterEntry` / `RegisterEntryRow` (which carry bsn/naam):
| field | type | notes |
|-------|------|-------|
| `id` | string (required) | zaak id — the entry's stable key |
| `status` | string (required) | enum `INGEDIEND` \| `INGESCHREVEN` (`RegistrationStatus`) |
| `reference` | string \| null | citizen-facing zaak identificatie (ADR-0012) |
`additionalProperties: false` so a record can't smuggle a field the schema didn't
sanction, and `dataClassification: "open"` records the intent that this objecttype is
public. **`bsn` and `naamPlaceholder` are deliberately absent** — public-safe by
construction, so S-19 cannot write a personal-data field into the public register even by
mistake.
### Registered over the API by a one-shot, not setup_configuration
The Objecttypen API's `setup_configuration` (3.4.2) provisions only tokens — it has no
declarative step to create an objecttype with a schema. So a `registerrecord-init`
compose one-shot (stdlib Python, on the stack network) creates the objecttype + a
**published** version over the API once Objecttypen is healthy, following the ADR-0020
self-seed pattern. It is **idempotent**: if a `RegisterRecord` with a version already
exists it is a no-op, so it is safe on every `up`.
- ponytail ceiling: no schema-migration/versioning story — a schema change means editing
`registerrecord.schema.json` and bumping the version by hand; the one-shot only ever
adds v1 if none exists.
- Upgrade path: if the schema evolves, have the one-shot diff the published schema and
POST a new version, or move to a declarative step once the upstream supports one.
## Consequences
**Positive**
- The public register's disclosure surface is fixed in one reviewed artifact
(`registerrecord.schema.json`) and enforced by Objecten's own validation.
- Self-seeds on a fresh `make up` / bare local compose; no manual step, no built image.
**Negative / costs**
- The public-safe field set now lives in two places — the BFF's `OpenbaarEntry` and this
schema — that must be kept in sync by hand (a drift check is a candidate for later).
- Hand-managed schema version (ceiling above).
## Coupling rules touched (CLAUDE.md §8)
None new. Registration talks to the Objecttypen API over its documented API. S-19 will
write records via the ACL (§8.1) — this ADR only fixes the schema they conform to.
@@ -0,0 +1,175 @@
# ADR-0028: Objecten holds the register, OpenZaak holds the process
- **Status:** Accepted
- **Date:** 2026-08-14
- **Deciders:** Respellion engineering
- **Slice:** S-19a (#149), first of the S-19 (#20) split
## Context
Until this slice the register existed only as a **derived** thing: the read projection
rows the Event Subscriber builds from NRC zaak notifications (ADR-0008). There is no
system anywhere that holds "who is registered" as a first-class record — drop the
projection database and the only way back is to replay ZGW history and re-derive it.
That is the wrong shape for a register. A BIG registration is a **fact about a person**
that outlives the case that produced it: it is looked up, corrected, superseded, and
retained on its own schedule. The zaak that produced it is a **process record** — it
opens, moves through statussen, and closes. Storing the fact inside the process record
(as zaak `eigenschappen`, the v1 placeholder PRD §"Registration" mentions) welds the two
lifecycles together: the register can then never be read, retained, or corrected without
going through the case system that happened to create it.
S-18 stood up Objecten + Objecttypen and registered the public-safe `RegisterRecord`
objecttype (ADR-0027). The open question this ADR closes: **where the authoritative
register record lives, and who writes it.**
## Decision
**The register record lives in the Objecten API as a `RegisterRecord` object. OpenZaak
keeps only the process. On approval the ACL writes both: the ZGW eindstatus, then the
register record.**
### Not zaak eigenschappen
Eigenschappen are per-zaaktype, untyped strings, and readable only by walking the zaak.
They inherit the zaak's lifecycle and its archiving regime, and they give the public
register no queryable surface of its own. Objecten gives a JSON-schema-validated record
(ADR-0027 makes that schema the disclosure boundary), a queryable collection, and a
lifecycle the zaak cannot drag around with it.
### The ACL writes it, not the domain or the Event Subscriber
CLAUDE.md §8.1 keeps upstream Common Ground modules behind the ACL. Objecten is such a
module, so the same rule applies: `ObjectenGateway` is the only code that talks to it,
and the domain keeps handing the ACL nothing but a zaak URL. The alternative — having the
Event Subscriber write the record when it sees the status notification — would make the
register a *second* derived artefact of ZGW, which is exactly the coupling this ADR
removes.
### Two writes, converging rather than transactional
Approval is now two writes across two modules, so it cannot be atomic. Both are made
idempotent instead:
- a ZGW status is an append-only log entry, so re-setting the eindstatus is harmless;
- the register write is an **upsert keyed on the zaak id** — search Objecten for an
existing object with that `id`, then PATCH it or POST a new one.
A caller that retries a half-failed approval therefore converges. This is the same
eventual-consistency posture as everywhere else in the system (CLAUDE.md §2.2, §8.6),
not an exception carved out for this path.
### The objecttype is resolved by name, lazily
The objecttype URL and version number are assigned by Objecttypen at seed time, so they
cannot be pinned in config — the ACL resolves them by the configured name
(`Acl__Objecten__ObjecttypeName`), taking the highest **published** version. This is the
same reasoning as ADR-0021 for zaaktypen.
Resolution happens on the first approval, not at startup, so the ACL needs no `depends_on`
on Objecten and will not crash-loop when it boots ahead of the seed. A failed resolution
is not cached, so it is retried on the next approval.
- ponytail ceiling: the resolution is memoised per gateway instance, and the gateway is a
transient typed `HttpClient` — in practice one extra GET per approval against a
neighbouring container.
- Upgrade path: lift it into a singleton cache (as `CachedZaaktypeCatalog` does for ZGW)
if approvals ever get hot enough for that GET to matter.
### The objecttype's UUID is pinned, not server-assigned
Objecten refuses to store an object whose objecttype it has not been configured with
(`ObjectType with url=… is not configured`), and its configuration identifies an
objecttype **by UUID** — supplied through a static `setup_configuration` file applied
when the container starts, before the `registerrecord-init` one-shot has run.
Rather than thread a seed-time UUID from one container into another's config, the UUID is
**pinned**: `infra/objecttypen-registerrecord/register.py` creates the objecttype with a
fixed UUID (the Objecttypen API accepts a client-supplied one), and
`infra/objecten/setup_configuration/data.yaml` declares that same UUID. Both sides are
declared up front, both stay idempotent, and neither has to wait for the other.
The cost is a constant duplicated across two files that must be kept in step; each carries
a comment pointing at the other.
### The ACL must reach Objecttypen at the URL Objecten knows it by
Objecttypen builds the `url` it returns from the request's own Host header, and Objecten
matches an incoming object's `type` against the `api_root` it was configured with. So an
ACL that reads Objecttypen at `http://localhost:8020` gets back a `localhost` objecttype
URL that Objecten then rejects as "not one of the available choices" — even though it is
the same objecttype.
`Acl__Objecten__ObjecttypenBaseUrl` must therefore match Objecten's configured
`api_root` (`http://objecttypen:8000/api/v2/`). This is the same class of constraint as
ADR-0006's "point the ACL at OpenZaak's container IP", and it is why the Objecten
integration tests only pass from inside the compose network.
### Objecten's notifications are off for this slice
Objecten publishes to a Notificaties API on every write, and `notifications_api_common`
**raises** rather than skipping when that configuration is absent — so with no NRC wiring,
every `POST /api/v2/objects` returns 500 after creating and rolling back the object.
Objecten → NRC is not wired: there is no broker, no Celery worker, no `objecten` kanaal and
no abonnement for it. Configuring only the client side would make writes succeed while
every message was dropped on the floor — a delivery path that looks wired and isn't. So
`NOTIFICATIONS_DISABLED` is set for Objecten in both compose files instead.
- ponytail ceiling: Objecten emits no notifications, so nothing downstream can react to a
register write yet.
- **Lifted by ADR-0029** (S-19b-1, #152): broker, worker, `objecten` kanaal and
notifications config now exist, and `NOTIFICATIONS_DISABLED` is `false`.
## Consequences
**Positive**
- The register is a first-class record with its own schema, lifecycle and query surface,
independent of the case that produced it.
- The disclosure boundary is enforced by Objecten's schema validation (ADR-0027), not by
discipline in projection code.
- The read projection can become a cache of Objecten rather than a re-derivation of ZGW —
done in S-19b-2 (#153), ADR-0030.
**Negative / costs**
- Approval writes to two modules and is eventually consistent; a failure between them
leaves a zaak in eindstatus without a register record until the approval is retried.
Nothing repairs that automatically yet.
- One more upstream module on the approval path, and one more dev credential
(`Acl__Objecten__Token`) in compose.
- Two new hand-kept constants: the pinned objecttype UUID (two files) and the objecttype
name (compose + `register.py`).
- ~~Until S-19b lands, the public register is still read from the NRC-derived projection, so
the register record is written but not yet read — the two must agree.~~ Closed by ADR-0030:
the projection is now derived from the register, so there is only one source to agree with.
## Coupling rules touched (CLAUDE.md §8)
None bent. §8.1 is extended in spirit — the ACL is the only code that talks to Objecten,
exactly as it is the only code that talks to ZGW. The domain still passes only a zaak URL,
and no service reaches Objecten's database.
## Verification
The end-to-end assertion lives in the Playwright happy path
(`tests/e2e/registration.spec.ts`, run by `verify-e2e`): after the behandelaar approves and
the openbaar register shows `INGESCHREVEN`, it asserts Objecten holds exactly one
`RegisterRecord` for *that* reference, with status `INGESCHREVEN` and no field outside the
public-safe schema.
It belongs there and not in `verify-domain`, which looks like the obvious home: that check
completes the Beoordelen task straight through Flowable REST (deliberately — it exists to
exercise the Workflow Client's REST contract), which bypasses the domain `decide` path that
calls the ACL. The e2e is the only check that drives a real approval.
`ObjectenGatewayIntegrationTests` (`Category=Integration`, so it runs under `verify-acl`
inside the compose network) drives the real gateway against a live Objecten + Objecttypen
pair: two writes for the same id leave exactly one object, carrying the second write's
status and nothing outside the public-safe schema.
All three findings above — the pinned UUID, the notifications block, and the base-URL
constraint — came out of running the gateway against those live modules while writing the
slice, not out of CI.
@@ -0,0 +1,122 @@
# ADR-0029: Objecten publishes register events to NRC
- **Status:** Accepted
- **Date:** 2026-08-14
- **Deciders:** Respellion engineering
- **Slice:** S-19b-1 (#152), first of the S-19b (#150) split
- **Supersedes in part:** ADR-0028's "Objecten's notifications are off for this slice"
## Context
ADR-0028 put the authoritative register record in the Objecten API and had the ACL write
it on approval. It also switched Objecten's notifications **off** — deliberately, with a
stated ceiling: there was no broker, no worker, no `objecten` kanaal and no abonnement, so
turning the client side on alone would have produced a delivery path that looks wired and
drops every message.
S-19b-2 (#153) wants the read projection sourced from register writes rather than
re-derived from ZGW zaak events. That needs the notifications to actually arrive. This ADR
builds the four missing pieces and lifts the ceiling.
## Decision
**Objecten publishes to the same NRC OpenZaak already publishes to, on the `objecten`
kanaal, delivered by its own Celery worker — provisioned declaratively on both sides,
exactly as ADR-0007 did for OpenZaak.**
- **Objecten** (`infra/objecten/setup_configuration/data.yaml`): a `zgw_consumers` service
`nrc` (api_type `nrc`) plus a `notifications_config` step naming it, and
`NOTIFICATIONS_DISABLED: "false"` in both compose files.
- **NRC** (`infra/opennotificaties/setup_configuration/data.yaml`): an `objecten` kanaal
alongside `zaken`.
- **`objecten-celery`**: a worker container on the Objecten image (`/celery_worker.sh`),
mirroring `oz-celery`, with `CELERY_BROKER_URL`/`CELERY_RESULT_BACKEND` on
`objecten-redis` db 1 (db 0 is already the cache).
### One NRC, one credential, one kanaal per publisher
Objecten reuses the `big-reference-seed` client OpenZaak publishes with. NRC verifies its
JWT and authorizes it against OpenZaak's Autorisaties API (ADR-0007), which grants that
client `heeft_alle_autorisaties` — so no second credential and no publisher-specific
authorization is needed. A second NRC, or a second credential, would buy isolation this
reference application has no use for.
The kanaal name is **not ours to choose**: the Objects API sends
`NOTIFICATIONS_KANAAL = "objecten"`. NRC rejects a publish to an unregistered kanaal
(`"Kanaal met deze naam bestaat niet"`), which is precisely what the failing check for this
slice reported first. Its filter set (`object_type`) matches the kenmerken the Objects API
sends, so an abonnement can narrow to one objecttype instead of receiving every write.
### Writers address Objecten as `objecten.local` — NRC rejects single-label hosts
NRC types a notification's `hoofdObject` and `resourceUrl` as DRF `URLField`s, so Django's
`URLValidator` runs on them — and it refuses a **single-label** host. Objecten fills both
from the object `url` that DRF built with `request.build_absolute_uri`, i.e. **the Host the
caller used**. Write to `http://objecten:8000` and NRC answers every publish with
```
{"hoofdObject":["Voer een geldige URL in."],"resourceUrl":["Voer een geldige URL in."]}
```
which `objecten-celery` then retries with exponential backoff, forever, in the background —
the write itself having returned 201.
`SITE_DOMAIN` does **not** fix this; it is not what builds those URLs. The fix is on the
caller side: the `objecten` service carries an `objecten.local` network alias, and every
component whose writes must be notified — the ACL (`Acl__Objecten__BaseUrl`), the gateway
integration tests, this slice's verify check — addresses it there. An alias rather than a
plain dotted `SITE_DOMAIN` so the host still **resolves in-network**: a subscriber that
follows `resourceUrl` reaches the record it points at, which S-19b-2 will do. Readers are
unaffected and keep using the plain service name.
This is the same class of constraint as ADR-0028's "the ACL's Objecttypen base URL must
match Objecten's configured `api_root`": these modules put request-derived hosts into data
another module then validates or dereferences.
- ponytail ceiling: nothing *enforces* that a new writer uses the alias — it would get a 201
and silently no notification.
- Upgrade path: if a second writer ever appears, rename the compose service to `objecten.local`
so the plain name stops working, rather than adding a lint.
### A worker, not a synchronous send
`notifications_api_common` only schedules the send on transaction commit. Without a worker
the task sits in redis forever and every register write is silently undelivered — the exact
half-wired state ADR-0028 refused to ship. No `beat` for Objecten: it is a publisher, not a
subscriber, and `nrc-beat` already drains NRC's delivery queue.
## Verification
`make verify-objecten-notifications` (`infra/run-objecten-notifications-check.sh`, in the
CI `verify-stack` job) registers an abonnement on the `objecten` kanaal pointing at a
throwaway webhook sink, writes a `RegisterRecord` exactly as the ACL does on approval, and
asserts the notification reaches the sink. That is the whole chain in one assertion:
Objecten → `objecten-celery` → NRC → `nrc-beat` → the callback. Any missing piece — broker,
worker, kanaal, notifications config — shows up as a non-delivery rather than as a green
config.
## Consequences
**Positive**
- A register write is now observable by anything that subscribes, which is what S-19b-2
(#153) needs to make the projection a cache of Objecten rather than a re-derivation of ZGW.
- ADR-0028's ceiling is lifted: the delivery path is proven end to end, not merely configured.
**Negative / costs**
- One more long-running container (`objecten-celery`) on an already memory-tight CI runner.
- A second publisher on the shared `big-reference-seed` credential — a credential rotation
now touches two modules.
- Objecten now has two in-network names, and which one a caller uses silently decides
whether its writes are notified (ceiling above).
- ponytail ceiling: notification delivery has no dead-letter or alerting — a failed publish
is visible only in the worker log.
- Upgrade path: if undelivered register events start mattering, subscribe an audit sink or
read NRC's own delivery admin rather than building a retry layer here.
## Coupling rules touched (CLAUDE.md §8)
None bent. This is infrastructure between two upstream modules, over their documented
APIs; no service reaches another's database. §8.6 (idempotency at every event boundary)
applies to whatever consumes the new kanaal — S-19b-2's problem, not this slice's.
@@ -0,0 +1,141 @@
# ADR-0030: The read projection is sourced from the register, not from ZGW
- **Status:** Accepted
- **Date:** 2026-08-28
- **Deciders:** Respellion engineering
- **Slice:** S-19b-2 (#153), second of the S-19b (#150) split
- **Builds on:** ADR-0008 (read projection store), ADR-0028 (Objecten holds the register), ADR-0029 (Objecten publishes to NRC)
## Context
ADR-0028 moved the authoritative register record into the Objecten API, and said what should
follow: "the read projection can become a cache of Objecten rather than a re-derivation of
ZGW." Until this slice it was still the latter — the Event Subscriber listened on the `zaken`
kanaal and inferred register state from case events:
- a `zaak`/`create` meant INGEDIEND;
- any `status`/`create` was taken to be the approval, so meant INGESCHREVEN — the subscriber
may not read OpenZaak (§8.1), so it could not tell one statustype from another;
- the citizen-facing reference was not in the notification at all, so every projection had a
second hop: ask the ACL for the zaak's identificatie (#78).
So the register — a fact about a person — was reconstructed by guessing at the lifecycle of the
case that happened to produce it. ADR-0029 made the register itself publish. This ADR switches
the projection over to it.
## Decision
**The Event Subscriber listens on the `objecten` kanaal and projects the `RegisterRecord` the
notification points at. The projection is a cache of the register; ZGW is no longer a source.**
- The subscriber's abonnement moves from `zaken` to `objecten` (`register-abonnement.py`, and
the CI projection check).
- An Objecten notification carries **no record data** — only the object URL and the objecttype
as a kenmerk — so the record is read back through the ACL (`POST /register-records/read`).
§8.1 applies to Objecten exactly as ADR-0028 established: the ACL is the only code that talks
to it.
- The accepted acties are `create`, `update` and `partial_update`. The last one is not
defensive breadth: the ACL upserts with PATCH, and DRF routes a PATCH through the notifying
`update()` while naming the action `partial_update` — which is what Objecten publishes. So
every approval arrives as `partial_update`, and accepting only `create`/`update` drops the
one state change this slice exists to project. `destroy` is deliberately not accepted:
removing a registration from the public register is its own decision.
- The record already carries `id`, `status` and `reference`, so the row is the record. The
zaak-shaped surface goes: `IsZaakCreated`, `IsZaakStatusSet`, `ZaakUrl`, `ZaakId`, and
`ToEntry`'s `Resource == "status"` inference are replaced by `IsRegisterRecordWritten` +
`ObjectUrl`, and the ACL enrichment hop disappears.
### The ACL writes an INGEDIEND record on submit
Before this slice only approval wrote a record, so re-sourcing alone would have silently
dropped every INGEDIEND row from the public register. `OpenZaakAsync` therefore upserts a
record with status INGEDIEND after opening the zaak, keyed on the same zaak id that approval
later upserts to INGESCHREVEN.
This is the same two-writes-converging posture ADR-0028 already accepted for approval, now on
the submit path too: both writes are idempotent, so a retried submit updates the record rather
than adding a second one (§8.6). The reference comes from the registration itself, so unlike
approval this path needs no ZGW read-back.
The alternative — a register holding only INGESCHREVEN — is arguably the more correct reading
of "public register", but it narrows what the openbaar portal shows and reads against PRD §68
("~50 register entries with diverse statuses"). Rejected as a behaviour change this slice was
not asked to make.
### The dedup key is the projected row, not the notification
NRC carries no notification id and may redeliver, so the idempotency key is derived from
content (as before). The obvious candidates both break here:
- **the object URL alone** — the ACL upserts *one object per registration*, so submit and
approval notify about the same URL, and the approval would be swallowed as a duplicate;
- **object URL + actie** — a retried approval is a second `update`, so it would be dropped
while genuinely being the same state (harmless), but a *third* distinct state would collide
with it (not harmless).
The key is therefore the object plus the state that write puts in the projection —
`objecten:object:{url}:{status}:{reference}`. A redelivery collapses; a genuine state change
does not. That is exactly the property §8.6 asks for, and it needs no version field from
Objecten's internals.
### The notification log holds the row, not the event
`processed_notifications` stops describing ZGW events (`actie`, `zaak_id`, `resource`) and
holds the projected row itself (`register_id`, `status`, `reference`). A rebuild becomes a
replay with no mapping rules and no upstream reads at all — §8.4 held before via the ACL hop;
now it holds outright.
The migration **drops** the old columns rather than renaming them. EF scaffolded renames
(`resource``register_id`, `zaak_id``status`) that would have carried ZGW values into
columns meaning something else entirely, and a rebuild would then have projected that garbage.
- ponytail ceiling: the migration empties both tables. A pre-slice row describes a zaak event
the new projector cannot reproject, and the registrations behind those rows have no
RegisterRecord in Objecten (only approvals wrote one), so they are not re-derivable from the
new source either.
- Upgrade path: fine while stacks are ephemeral. If a long-lived environment ever needs to keep
them, backfill by walking Objecten's objects rather than replaying the log.
## Consequences
**Positive**
- The register is read from the register. The projection is a derived cache of a first-class
record, not an inference over someone else's lifecycle.
- The "any status-create is the approval" guess is gone — a real source of wrongness the moment
the zaaktype grows a second statustype.
- One hop fewer per notification: the record carries its own reference, so the ACL enrichment
call disappears.
- A rebuild needs nothing but its own log (§8.4).
**Negative / costs**
- Submission is now two writes across two modules and eventually consistent. A failure between
them leaves a zaak with no register record until the submit is retried; nothing repairs that
automatically yet — the same gap ADR-0028 recorded for approval, now on a second path.
- The projection lags the register by a notification round trip, where it used to lag the zaak
by one. In practice the same order of magnitude.
- Projecting now depends on the ACL being reachable, where the reference enrichment used to be
the only ACL dependency. A failed read means the notification is not logged and not
projected — NRC retries, so it converges, but the failure mode is now on the main path.
- OpenZaak still publishes to `zaken` and nothing in the product listens. Kept because the
`verify-nrc` check asserts that path, and turning off a working publisher to save nothing
would be its own risk.
## Coupling rules touched (CLAUDE.md §8)
None bent. §8.1 holds — the subscriber reaches Objecten only through the ACL. §8.4 is
strengthened: the projection is rebuildable from its own log, with no upstream reads at all.
§8.6 is what the dedup-key discussion above is about.
## Verification
`make verify-projection` (`infra/run-projection-check.sh`, in CI's `verify-stack`) opens a zaak
**through the ACL** and asserts projection-api serves a row for it with status INGEDIEND — the
whole new chain in one assertion: ACL → Objecten → `objecten-celery` → NRC → `nrc-beat`
Event Subscriber → projection → projection-api. A zaak created behind the ACL's back produces
no row, which is the re-source working rather than a gap.
`RegisterProjectieBijwerken.feature` covers the use case in business language, including the
approval case — the same row moving INGEDIEND → INGESCHREVEN, which is now one registration's
record being updated rather than two unrelated ZGW events.
@@ -0,0 +1,49 @@
# ADR-0031 — MFA on the medewerker realm, with a fixture TOTP secret
- **Status:** Accepted
- **Date:** 2026-09-03
- **Slice:** S-15c (Gitea #132)
## Context
Staff (behandelaar, teamlead, beheerder) act on citizens' registrations and on the ACL's
default-fill: the highest-privilege logins in the platform. The medewerker realm protected
them with a password alone, while the citizen realms (digid, eherkenning, eidas) mock
brokers that carry their own assurance levels. A reference application that demonstrates a
government architecture should show MFA on the staff realm.
Two things had to be decided: **how** to enforce OTP in a realm export, and **how the
automated checks and a human demo obtain a code** — the e2e drives a real browser login and
`make keycloak-smoke` drives a real password grant, so neither can scan a QR.
## Decision
**Enforce OTP by giving every seeded medewerker a TOTP credential**, rather than replacing
Keycloak's browser flow with a copy whose OTP execution is `REQUIRED`.
Keycloak's stock `browser` and `direct grant` flows both contain a *conditional OTP*
subflow that fires when the user has an OTP credential. Seeding the credential therefore
turns the challenge on for every seeded user, in both flows, without duplicating ~40 lines
of flow JSON into the export. `CONFIGURE_TOTP` is additionally set as a **default required
action**, so a medewerker created later must enrol before their first login.
**The seeded secret is a fixed, committed fixture** (`BIGMEDEWERKEROTPSEED`) shared by all
medewerkers. Codes are then computable: `infra/keycloak/check_realms.py` (Python, stdlib
`hmac`) and `tests/e2e/medewerker-login.ts` (Node `crypto`) each implement RFC 6238 in
about six lines — no OTP dependency on either side, and no enrolment step in the tests.
## Consequences
- A password alone no longer yields a token on the medewerker realm; `check_realms.py`
asserts that refusal, so the enforcement cannot silently regress.
- Every medewerker login in the e2e goes through `loginMedewerker()`, which submits the OTP
form. New staff specs must use it.
- **The secret is public.** It is a demo fixture and worthless outside this synthetic
stack, in the same class as the committed `test123` passwords and the mock DigiD broker.
A real deployment enrols per-user authenticators (or federates to DigiD Machtigen /
eHerkenning at the required assurance level) and seeds no credentials at all.
- Enforcement is *effectively* realm-wide but *technically* per-user: the conditional
subflow is what fires. A medewerker whose OTP credential were removed would fall back to
the required action at next login (enrol, then challenge) rather than skipping MFA — an
acceptable equivalence for this purpose, and the reason the required action is set.
- Reversal is a one-file edit: drop the `otp` credentials and the `requiredActions` block.
@@ -0,0 +1,79 @@
# ADR-0032: The werkbak refreshes itself by polling, not by a pushed stream
- **Status:** Accepted
- **Date:** 2026-09-04
- **Deciders:** Respellion engineering
- **Slice:** #162 (proposal #163). The issue titles it S-26; that id already belongs to
the self-service resume slice (#111), so #162 is the identifier that counts.
## Context
The werkbak (S-12) is a read of the open Flowable `Beoordelen` tasks: portal → BFF
`GET /behandel/werkbak` → domain `Werkbak` query → workflow engine, each task enriched
from its aggregate. A registration reaches `Beoordelen` **asynchronously**, only once the
citizen supplies its documents and the DMN routes it (S-10a) — so it appears in a werkbak
that is already open, and until now a behandelaar had to reload the page to see it.
Three forces shape the mechanism:
- **Nothing notifies anyone.** The trigger lives in Flowable. The domain does not publish
task events, and there is no bus between the domain and the BFF.
- **The BFF is stateless** and sits behind each portal's reverse proxy.
- **This is the repo's first live-updating view**, so the choice sets a precedent.
## Decision
**The werkbak page re-reads the existing BFF endpoint on a fixed interval
(`WERKBAK_REFRESH_MS`, 5 s) while it is open. No new endpoint, dependency or server-side
state.**
The refresh is a *background* read: it leaves the rows and the loading/failure states
untouched until it has an answer, so a tick never flashes a spinner over rows a
behandelaar is reading and a single failed poll never swaps the list for the error alert.
A read that comes back also clears an earlier failure, so the view recovers on its own —
the same reload this slice set out to remove would otherwise be needed to escape a
transient error. Only a foreground read (on open, after a decision) speaks for whether the
werkbak is readable at all.
### Why not SSE or WebSockets
Neither buys freshness here, because **nothing notifies the BFF either**:
- **SSE** (`text/event-stream`) would mean a new streaming endpoint whose handler polls the
domain and forwards diffs — the same latency, plus connection lifecycle, proxy
buffering, and auth on a long-lived connection.
- **WebSocket/SignalR** adds a dependency (CLAUDE.md §13) and makes the BFF stateful and
sticky-session-bound. A genuine push path would *also* need the domain to publish task
events. Warranted by high-frequency, bidirectional or fan-out-heavy traffic; the werkbak
is none of those.
Polling meets the acceptance ("a registration can be seen in the werkbak once it is ready
for review") in a handful of lines inside one component.
- ponytail ceiling: a fixed 5 s interval, per open page, that keeps polling in a
background tab. Each tick costs one Flowable task query plus a store read per open task.
- Upgrade path: publish task events from the domain, then swap the component's `interval`
for a stream. The endpoint contract and the component's rendering stay as they are;
gate on `document.visibilityState` first if request volume is the concern.
## Consequences
**Positive**
- The outcome is delivered with no new endpoint, dependency, or server-side state, and no
service boundary moves.
- Self-healing: a transient read failure no longer strands the view until a manual reload.
- The e2e got *simpler* — the happy path waits for the werkbak row without reloading the
page, which is itself the live-refresh assertion.
**Negative / costs**
- Staleness is bounded by one interval (≤5 s) rather than instant.
- One `GET /behandel/werkbak` per open werkbak per interval, including in hidden tabs.
- The precedent is polling; a future view with genuinely high-frequency updates will have
to revisit this (see the upgrade path above).
## Coupling rules touched (CLAUDE.md §8)
None. The poll reuses the existing portal → BFF → domain read path: §8.3 (portals talk
only to the BFF) and §8.2 (only the Workflow Client talks to Flowable) are unchanged.
@@ -0,0 +1,162 @@
# ADR-0033: Kubernetes deployment is one values-driven Helm chart, not a chart per service
- **Status:** Accepted
- **Date:** 2026-09-04
- **Deciders:** Respellion engineering
- **Slice:** #25 (S-24) — raised directly as a deployment-target request and matched to
that issue afterwards; see the "Process note" at the end
## Context
The stack is defined once, in `infra/docker-compose.yml`: 30-odd containers made of six
upstream Common Ground modules (OpenZaak, Open Notificaties, Objecten, Objecttypen,
Keycloak, Flowable), their databases and workers, five .NET services, four portals, six
one-shot bootstrap containers, and an observability backplane (off by default here). Compose is the
CI-canonical stack: `make verify` and every `verify-*` script drive it.
We now also want the stack on Kubernetes — first target a **single-node Talos VM on a
laptop**. Four properties of this particular stack shape the answer:
- **The upstream images are used verbatim** and read their configuration from a mounted
directory (`setup_configuration/data.yaml`, Keycloak realm exports, BPMN/DMN). Compose
streams those files into external volumes (`infra/seed-config.sh`) because bind mounts
don't reach sibling containers on the CI runner. Kubernetes needs the same files as
ConfigMaps — from *somewhere*.
- **Django's `URLValidator` rejects single-label hosts.** Compose works around it by
handing the ACL and the seeds a container *IP* (ADR-0009, ADR-0020, ADR-0029, and the
`objecten.local` network alias). In Kubernetes a Service FQDN is already multi-label, so
the workaround has a natural replacement — but the hosts have to line up exactly, since
Objecten reflects the request Host into the URLs it publishes to NRC.
- **The OIDC issuer must be one string** for both the browser and the BFF (ADR-0010).
`infra/host-browser.yml` already solved this for a host browser: pin `KC_HOSTNAME`, keep
backchannel discovery in-cluster, and mount a `config.json` per portal.
- **Nothing here is highly available.** One replica of everything, on one node.
## Decision
**One chart — `infra/helm/big-reference` — whose `values.yaml` is a near-literal
transcription of the compose file, rendered by three generic templates (Deployment, Job,
Service) over a `workloads` map.** Adding a service is a values edit.
Consequences of that shape, each chosen deliberately:
- **Config files are not copied into the chart.** `infra/helm/seed-configmaps.sh` creates
the ConfigMaps from the files that already live in the repo — the Kubernetes sibling of
`infra/seed-config.sh`. The chart therefore needs `make k8s-seed` before `helm install`,
which is the same two-step dance compose already has.
- **Bootstrap one-shots become Jobs, with no ordering mechanism.** Every one is idempotent
(ADR-0020); each waits for the TCP ports it needs via a busybox init container and
Kubernetes retries the rest. `make k8s-reseed` re-runs them.
- **The four Django services apply their own `setup_configuration`**
`args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]` — instead of getting a
separate `*-init` Job like compose. Both of those image scripts run
`manage.py migrate`, and compose serialises them with
`depends_on: service_completed_successfully`; Kubernetes has no such edge, so a Job and
its web pod migrate the same database concurrently and Django dies with
*"relation zgw_consumers_service already exists"*. Running the two steps in order inside
the one container leaves exactly one migrator per database, and deletes four workloads.
- **`args`, never `command`.** Compose's `command:` replaces the image's CMD; Kubernetes'
`command:` replaces its ENTRYPOINT. Transcribing one to the other silently broke every
upstream image that relies on its entrypoint — postgres ran as root and refused to
start, Keycloak tried to exec `start-dev` as a binary. The chart now `fail`s at render
time if a workload sets `command`, because the symptom (a crashloop three layers down)
is nothing like the cause.
- **Published ports are NodePorts.** No ingress controller, no LoadBalancer, no TLS. The
four portals are the exception in *use*, not in wiring: PKCE needs `crypto.subtle`, which
browsers expose only in a secure context, so a portal has to be reached over `localhost`
(`make k8s-portals` forwards them) or eventually over HTTPS. `.Values.host` is therefore
"the address the browser uses", not "the node's address" — it pins Keycloak's issuer and
each portal's `config.json`, and both must agree with the URL bar (ADR-0010).
- **Databases are `emptyDir` by default**, so the stack comes up on a cluster with no CSI
driver; setting `persistence.storageClass` switches every database to a PVC.
- **Only two hosts become FQDNs** — OpenZaak (for the ACL and the zaaktype seed) and
Objecten (for the ACL's register writes), the two that Django validates as URLs.
Everything else keeps the short compose service name, because the upstream
`setup_configuration` files name those and Objecten matches an objecttype URL against the
one it was configured with. The portals used to be a third case — nginx's `resolver` never
appends search domains, so the bare `bff` upstream could not resolve on Kubernetes — which
ADR-0034 removed by serving them with Caddy, whose resolver honours `/etc/resolv.conf`.
- **Compose stays CI-canonical.** The chart is a second deployment target, not a
replacement; the acceptance, verify and e2e lanes are unchanged.
### Alternatives considered
- **A chart per service, or an umbrella of 30 subcharts.** The conventional layout, and
roughly 1,500 lines of near-identical YAML for a stack where 28 of 30 workloads are
"one pod, one image, some env". It buys independent versioning we don't want (the stack
is demoed as a whole) and costs the eye-diffability against the compose file that keeps
the two stacks honest.
- **`kompose convert`.** One-shot generation, no ongoing artefact to maintain — but it
drops exactly the parts that carry the design (init ordering, the config volumes, the
issuer pinning) and produces output nobody owns.
- **Bitnami PostgreSQL/Redis subcharts.** Six more dependencies (CLAUDE.md §13) and a
second way of expressing the same three-line database.
- **ingress-nginx with hostname routing.** Needs a controller, `/etc/hosts` entries and a
matching issuer host; NodePorts need none of it and reuse the mechanism
`infra/host-browser.yml` already proves.
- **A registry on the laptop** (the obvious home for images built there). Talos cannot
side-load an image, so a registry is required either way — but reaching one on the host
means opening an inbound port on firewalld's `libvirt` zone, which needs root, and
pushing to it over plain HTTP means an `insecure-registries` entry in the Docker daemon,
which needs root again. `infra/helm/registry.yaml` runs the registry *in* the cluster on
a NodePort instead: pushing laptop → node is outbound and unfiltered, the node pulls from
its own NodePort, and `docker save | crane push --insecure` needs no daemon
configuration. Cost: one more (throwaway, `emptyDir`) workload, and a re-push if its pod
is replaced.
- **Helm hooks (`pre-install`/`post-install`) for bootstrap ordering.** Hooks run after
`--wait`, which would deadlock: OpenZaak's readiness needs the migrations that the hook
is supposed to run. Idempotent Jobs plus retries need no such sequencing.
- ponytail ceiling: single-node assumptions are baked in — one replica per workload,
`Recreate` rollouts, ReadWriteOnce volumes, no PodDisruptionBudgets, no resource
requests or limits (a laptop VM schedules everything or nothing), plain HTTP.
Upgrade path for a real cluster: add requests/limits per workload (the field is already
passed through), swap NodePorts for an Ingress with TLS, and give the databases a real
StorageClass — none of which changes the workload graph.
## Consequences
**Positive**
- One file to read to see what the cluster runs, and it lines up with the compose file
line for line.
- The compose IP workarounds disappear: cluster DNS supplies multi-label hosts.
- `make k8s-lint` renders and schema-checks the whole stack without a cluster.
- The config inputs have exactly one home (the repo) for both stacks — no fork to drift.
**Negative / costs**
- A second deployment description to keep in step with compose. `make k8s-drift` (#168)
now enforces the part that bites — the workload set and the resolved images, with the
four deviations below declared — but not per-workload env, ports or volumes.
- `helm install` alone is not enough — the ConfigMaps must be seeded first, and a missing
one surfaces as `ContainerCreating`, not as a clear error.
- Generic templates mean a values typo can render valid-but-wrong YAML; `k8s-lint` catches
schema errors, not intent.
- The verify/e2e lanes do not run against the chart, so the Kubernetes path is verified by
hand (docs/runbooks/kubernetes-talos.md §5) rather than by CI.
- The chart deviates from compose in four places now (args, self-configuring Django pods,
FQDN hosts, NodePorts). Each is forced by the platform and commented where it appears,
but it is four more things that can drift.
## Coupling rules touched (CLAUDE.md §8)
None. The chart deploys the same graph: portals reach only the BFF (§8.3), only the ACL
holds ZGW credentials (§8.1), only the Workflow Client talks to Flowable (§8.2), each
service keeps its own database (§8.5). No workload gained a peer it didn't have in compose.
## Verified
Brought up from scratch on a single-node Talos v1.14.0 VM (6 vCPU / 10 GB, virtio disk)
under virt-manager: 29 pods ready and four bootstrap Jobs complete in under three minutes,
with zero restarts, using ~4.4 GB of the VM's 10 GB. The smoke test in the runbook's §5
walks the whole path — portal proxy → BFF → domain → Flowable → ACL → OpenZaak + Objecten →
NRC → event-subscriber → projection → public register — plus a werkbak read with an
MFA'd medewerker token. The browser flow itself was driven with Playwright against
`http://localhost:30140`: secure context, PKCE, Keycloak form, login, no console errors.
## Process note
CLAUDE.md §14 wants the ADR proposal issue opened before the code, and §7 wants a slice
issue behind the work. This landed the other way round — chart first, on request. The
issue and the CI drift check are the outstanding follow-ups.
@@ -0,0 +1,103 @@
# ADR-0034: The portals are served by Caddy, not nginx
- **Status:** Accepted
- **Date:** 2026-09-04
- **Deciders:** Respellion engineering
- **Slice:** _(none yet — raised directly alongside the Kubernetes deployment, ADR-0033)_
## Context
Each portal ships as one image that does two jobs: serve the built Angular app, and
reverse-proxy *its own* BFF endpoint group so the browser calls a single origin (no CORS,
and the DigiD/medewerker token rides along — ADR-0010, ADR-0013). Until now that was nginx
with a hand-written `nginx.conf` per app.
Two workarounds had accumulated around nginx's resolver, both for the same root cause:
**nginx resolves a variable `proxy_pass` upstream itself**, using only the `resolver`
directive, and never the search domains in `/etc/resolv.conf`.
1. `resolver 127.0.0.11` (Docker's embedded DNS) is wrong on rootless podman, which uses a
network-specific aardvark address — so `apps/portal-nginx-resolver.sh` rewrote the
directive at container start by reading the pod's actual nameserver.
2. On Kubernetes the bare `bff` name cannot resolve at all without the `svc.cluster.local`
search domain, so the same script gained a `BFF_HOST` override that the Helm chart set
per portal (ADR-0033).
Both existed only to tell the proxy how to resolve one hostname.
## Decision
**Serve the portals with `caddy:2-alpine` and a small `Caddyfile` per app, replacing the
nginx runtime stage, the four `nginx.conf` files, and the resolver workaround.**
Caddy dials its upstream per request through Go's resolver, which reads
`/etc/resolv.conf` — nameserver *and* search domains. So `reverse_proxy bff:8080` resolves
correctly under Docker, rootless podman and Kubernetes with no per-engine configuration,
and it still starts before the BFF exists and picks up its restarts (the property the
variable `proxy_pass` was there to buy). `apps/portal-nginx-resolver.sh`, its unit test and
the chart's `BFF_HOST` env are deleted.
The Caddyfile uses `handle` blocks rather than a bare `try_files`:
```
handle /behandel/* { reverse_proxy bff:8080 }
handle { root * /usr/share/caddy; try_files {path} /index.html; file_server }
```
`handle` blocks are mutually exclusive and matched most-specific-first. This matters:
Caddy's default directive order puts rewrites (`try_files`) *before* `reverse_proxy`, so a
top-level `try_files {path} /index.html` would rewrite every API path to `/index.html`
before the proxy ever saw it — the SPA fallback would silently eat the API. The `handle`
form makes the routing explicit instead of relying on directive-order trivia.
`infra/test_portal_caddyfiles.py` (in `make unit`) asserts each portal proxies exactly its
own endpoint groups and keeps the SPA fallback. The four files are near-identical, so a
copy-paste slip is cheap to make and expensive to find: proxying another portal's group
hands a browser an endpoint its token isn't for, and the failure surfaces as a 401 three
services away.
### Alternatives considered
- **Keep nginx.** Zero migration, and it works — but the resolver workaround stays, and it
had already grown a second head for Kubernetes. Both heads are nginx-specific.
- **Keep nginx, hard-code the FQDN.** Would need a different config per deployment target
(compose vs Kubernetes), which is exactly the fork the chart was written to avoid.
- **Drop the proxy and use CORS.** Turns the same-origin design (ADR-0010) inside out:
CORS preflights, an explicit origin allowlist in the BFF, and a token attached
cross-origin. Not a serving decision — an architectural regression.
- **Kubernetes Ingress in front of the portals.** Solves nothing about compose, adds a
controller, and the portals would still need something to serve static files.
- ponytail ceiling: plain HTTP on `:80`, no compression, no cache headers beyond Caddy's
defaults, and Caddy's automatic HTTPS deliberately unused (there is no hostname to get a
certificate for). Upgrade path: `encode zstd gzip` and a cache policy for immutable
Angular bundles; a real hostname makes TLS a one-line `Caddyfile` change, which is the
main reason this is worth having in place.
## Consequences
**Positive**
- One resolver behaviour across compose, podman and Kubernetes; a script, a unit test and a
chart env var are deleted rather than maintained.
- The images gain `curl` for free (the alpine nginx image had only busybox `wget`), which
the compose healthchecks can use.
- Routing intent is readable: one `handle` block per endpoint group, one for the app.
- TLS later is a one-line change instead of a new component.
**Negative / costs**
- A new runtime dependency in four images (CLAUDE.md §13): Caddy replaces nginx rather than
joining it, so the count is unchanged, but it is a less familiar config language for
anyone who has only read nginx configs.
- The images grew: 90.6 MB against nginx's 75.7 MB, because `caddy:2-alpine` carries a
bigger static binary than nginx's. Measured, not estimated.
- Caddy's directive-order rule is a genuine footgun (see above); the `handle` form and the
Caddyfile comments exist to keep the next person out of it.
- Any operational note that says "the portal's nginx" is now wrong; the ones in `docs/` were
updated with this ADR.
## Coupling rules touched (CLAUDE.md §8)
None. §8.3 is unchanged and unchanged in kind: the portals still talk only to the BFF, and
the proxy is still the thing that makes that same-origin.
+50
View File
@@ -0,0 +1,50 @@
# FDS-architectuur — Open Register
Deze map bevat de architectuurbesluiten en de engineer-documentatie voor de FDS-kant van deze
referentie-applicatie: deelnemen aan het Federatief Datastelsel als **afnemer**.
De strategische inzet, de slices en de portfoliostatus staan in het Innovation Lab-repo,
`Respellion/innovation-lab`, onder `projects/open-register-fd/`. Daar staan ook de
architectuurblauwdruk, de FDS gap-analyse en de privacy-views.
## Documenten
| Document | Waarvoor |
|---|---|
| [`c4-component-view.md`](c4-component-view.md) | Componentview op niveau 3: ports en adapters, en welke views nog waarde toevoegen |
| [`slice-1-proposal.md`](slice-1-proposal.md) | Het bouwbare eerste increment; plak dit in een `poc-voorstel`-issue |
| `adr/` | De geaccepteerde architectuurbesluiten, ADR-0001 tot en met ADR-0006. Zie de tabel hieronder. |
## Architecture Decision Records
Een ADR legt een besluit vast dat **vaststaat**, met de context en de gevolgen, zodat het niet stil
opnieuw wordt uitgevochten. Statuswaarden: `proposed``accepted` → (`vervangen door ADR-NNNN` |
`deprecated`).
Een geaccepteerde ADR wijzigen betekent een nieuwe ADR schrijven die de oude vervangt. Wij
herschrijven de historie nooit.
ADRs liggen naast governance. Acceptatie volgt de asynchrone bezwaarronde uit
`Respellion/innovation-lab`, `operating-model/operating-model.md`, sectie *Besluitvorming*.
| ADR | Besluit | Status |
|---|---|---|
| [0001](adr/0001-acl-at-every-register-boundary.md) | Anti-Corruption Layer op elke registergrens | accepted |
| [0002](adr/0002-fsc-for-connectivity.md) | FSC voor connectiviteit tussen organisaties, geen ruwe REST | accepted |
| [0003](adr/0003-pbac-via-opa.md) | Policy-based access control via OPA, FTV-klaar | accepted |
| [0004](adr/0004-bounded-cache.md) | Begrensde cache; registers blijven systeem van registratie | accepted |
| [0005](adr/0005-ldv-verwerkingenlog.md) | Verwerkingenlog via event-emissie, in lijn met LDV | accepted |
| [0006](adr/0006-module-boundary-and-reuse.md) | Modulegrens en hergebruikstrategie: in-process → .NET-module → OpenMetadata-feed → gateway op verzoek | accepted |
## Nummering
Deze reeks staat los van de ADR-reeks over de referentie-applicatie zelf, die in
[`../`](../adr-0001-loose-coupling.md) loopt van `adr-0001-loose-coupling` tot en met
`adr-0010-bff-oidc`. Vandaar de eigen map `fds/`: beide reeksen beginnen bij 0001, en de nummers
zouden anders over de volle breedte botsen.
In de MkDocs-navigatie staan deze zes daarom als **FDS ADR-000N**, zodat de zijbalk ze niet met de
reeks van de applicatie verwart.
Nieuwe FDS-ADR: kopieer [`adr/template.md`](adr/template.md), neem het volgende nummer, en open een
pull request.
@@ -0,0 +1,42 @@
# ADR-0001: Anti-Corruption Layer op elke registergrens
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle (Build, Lead Link)
- **Vervangt / vervangen door:**
## Context
De applicatie bevraagt meerdere registers: BRP, NHR/KVK, en ZGW via OpenZaak. Hun vocabulaires en
schema's verschillen van elkaar en van ons domein. Zij veranderen ook zelf mee met de FDS-standaarden.
Lekt registervocabulaire het domeinmodel in, dan werkt elke wijziging aan de registerzijde door in de
bedrijfslogica. Het domein wordt dan een lappendeken van vreemde begrippen in plaats van ubiquitous
language.
## Besluit
Elk register is bereikbaar via een Anti-Corruption Layer: **één adapter per register**, die een
**port** vervult die het domein definieert.
Adapters doen alleen vertalen en velden versmallen. Zij bevatten geen bedrijfslogica. Het domein
spreekt `Persoon` en `Organisatie`, en nooit veldnamen uit BRP of NHR.
## Gevolgen
**Positief:** verloop in registers en FDS-standaarden blijft bij de adapter. Het domein blijft stabiel
en testbaar. Adapters zijn onafhankelijk vervangbaar, en dat is precies wat de FSC-wissel uit
ADR-0002 goedkoop maakt. Het patroon generaliseert naar een herbruikbare ACL-template per register,
een Foundations-kandidaat.
**Negatief en kosten:** één vertaalmap per register om te schrijven en te onderhouden, plus een extra
indirectie die engineers moeten respecteren in plaats van omzeilen.
**Vervolgwerk:** extraheer de ACL-template zodra de tweede adapter bestaat (slice 3).
## Overwogen alternatieven
- **Registers direct aanroepen uit de applicatieservices** — afgewezen: dit koppelt bedrijfscode aan
registerschema's en aan versies van FDS-standaarden.
- **Eén generieke registeradapter** — afgewezen: registers verschillen genoeg dat een generieke
abstractie zou gaan lekken of opzwellen. Adapters per register zijn duidelijker.
@@ -0,0 +1,44 @@
# ADR-0002: FSC voor connectiviteit tussen organisaties, geen ruwe REST
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle, Upstream Liaison
- **Vervangt / vervangen door:**
## Context
Registerbevragingen kruisen een organisatiegrens naar systemen van bronhouders met
persoonsgegevens. Het FDS noemt Federatieve Service Connectiviteit (FSC, de opvolger van NLX) als de
richting voor connectiviteit: wederzijdse authenticatie op organisatieniveau, autorisatie
gecontroleerd tegen een contract en gehandhaafd bij de bron, en symmetrische transactielogging.
Een ruwe REST-client met mTLS geeft ons geen van de contractadministratie, delegatie of onafhankelijke
tweezijdige verantwoording die een FG of auditor nodig heeft.
## Besluit
Het FSC Client-component stuurt alle registerbevragingen via een **FSC outway**, de
EUPL-referentie-implementatie. De ACL-adapter hangt af van de FSC Client, en niet van een HTTP-client.
FSC-zaken — contracten, identiteiten, delegatie — leven in dit component, achter de Register Port.
## Gevolgen
**Positief:** de autorisatie wordt bij de bron gehandhaafd, en niet op gezag van de aanroeper
vertrouwd. Onweerlegbaar loggen aan beide uiteinden maakt onafhankelijke afstemming tegen ons LDV-log
mogelijk. Delegatie wordt expliciet meegedragen. Wij lopen in lijn met de FDS-richting, vóór er een
verplichting is.
**Negatief en kosten:** FSC is operationeel zwaarder dan een REST-aanroep — beheer van certificaten en
identiteiten, plus een outway die op De Werf moet draaien. De vergelijking FSC tegenover DSP loopt
binnen het FDS nog, dus sommige details kunnen schuiven.
**Vervolgwerk:** valideer het contract- en logginggedrag van de huidige fsc-nlx-implementatie
(slice 2). Herzie dit als het FDS voor DSP kiest; ADR-0001 houdt die wissel beperkt tot één component.
## Overwogen alternatieven
- **Ruwe REST met mTLS** — afgewezen: geen contractlaag, geen tweezijdig log, en het wijkt af van het
FDS.
- **Wachten tot het FDS FSC tegenover DSP heeft beslist** — afgewezen: de naad uit ADR-0001 laat ons nu
adopteren en later aanpassen. Wachten geeft het voordeel van vroege expertise weg.
@@ -0,0 +1,44 @@
# ADR-0003: Policy-based access control via OPA, FTV-klaar
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle, FG (geconsulteerd)
- **Vervangt / vervangen door:**
## Context
Elke bevraging van persoonsgegevens uit BRP of NHR is een verwerking die een grondslag en een
begrensde doelbinding nodig heeft. Toegangsregels moeten handhaafbaar en auditeerbaar zijn, en
wijzigbaar zonder de bedrijfscode opnieuw uit te rollen.
De Federatieve Toegangsverlening (FTV) van het FDS beweegt naar policy-based access control, maar is
nog geen afgeronde standaard.
## Besluit
Introduceer een Policy Decision Point met Open Policy Agent (OPA). De applicatieservices roepen de
PDP aan — via een Authorisation Port en een PDP Client — **vóór elke registerbevraging**, en geven
rol, doel en grondslag mee.
Policies schrijven wij als code, **geversioneerd in Gitea**, en zij gaan via review naar productie. De
PDP staat zo gepositioneerd dat wij bij de komst van FTV alleen het policy-dialect opnieuw uitdrukken,
zonder de architectuurgrens te verplaatsen.
## Gevolgen
**Positief:** doelbinding en grondslag worden gehandhaafd, en niet alleen gedocumenteerd. De FG kan de
werkelijke regels in versiebeheer lezen, waardoor het verwerkingenregister en de gehandhaafde policy
naar elkaar toe groeien. Toegangswijzigingen zijn reviewbaar en gedateerd.
**Negatief en kosten:** BRP-autorisatiebesluiten correct modelleren is juridisch werk, geen
engineering. De PDP maakt de handhaving betrouwbaar, niet de policy juist. Daarnaast komt er een
component bij om te exploiteren.
**Vervolgwerk:** een promotiepijplijn voor policies in Gitea Actions. Policies opnieuw uitdrukken zodra
FTV stabiliseert. Een FG-review van de policy-set vóórdat er echte persoonsgegevens in komen.
## Overwogen alternatieven
- **Rolcontroles in de applicatiecode** — afgewezen: niet auditeerbaar, niet wijzigbaar zonder deploy,
en het verspreidt toegangslogica over de codebase.
- **Wachten op FTV** — afgewezen: de PBAC-vorm is al duidelijk. Nu OPA, later het FTV-dialect.
@@ -0,0 +1,48 @@
# ADR-0004: Begrensde cache; registers blijven systeem van registratie
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle, FG (geconsulteerd)
- **Vervangt / vervangen door:**
## Context
*Data bij de bron* verbiedt het behandelen van registerdata als lokale bron van waarheid. Maar BRP of
NHR bij elke interactie bevragen is onpraktisch en vergroot de blootstelling.
Persoonsgegevens zijn de data die wij het minst willen opbouwen. Een onbegrensde cache wordt stil een
schaduwregister, met een onbeheerde bewaarverplichting als gevolg.
## Besluit
Een **begrensde cache** staat achter een Cache Port, beheerd door een Cache Manager. Vier grenzen
gelden.
| Grens | Wat die betekent |
|---|---|
| **Tijd** | Een TTL die aan het doel hangt |
| **Omvang** | Alleen de werkset van een actieve zaak |
| **Gezag** | Antwoordt nooit wat de bron niet zou antwoorden; geen systeem van registratie |
| **Adresseerbaarheid** | Gesleuteld op subject, zodat verwijderen op verzoek kan |
Purge-triggers: het verstrijken van de TTL, het sluiten van de zaak, en een verwijderingsverzoek.
## Gevolgen
**Positief:** de prestaties van een lokale kopie, zonder een onbevoegd register te worden. Bewaartermijn
en het recht op verwijdering zijn echte operaties, geen hoop. Dit is consistent met zowel
AVG-dataminimalisatie als FDS-data-bij-de-bron.
**Negatief en kosten:** de mapping van doel naar TTL is een beleidsbesluit, samen met de FG en de
autorisatievoorwaarden, en geen engineeringconstante. Die is dus makkelijk fout te krijgen. Daarnaast
komt de complexiteit van cache-invalidatie erbij.
**Vervolgwerk:** definieer het beleid voor doel naar TTL met de FG. Maak een toestandsdiagram voor de
levensloop van een cache-entry. Documenteer de aanvaardbare veroudering per register.
## Overwogen alternatieven
- **Geen cache; altijd de bron bevragen** — afgewezen: onpraktische latency en belasting, en meer
blootstelling per aanroep.
- **Een onbegrensde of algemene cache** — afgewezen: die wordt een schaduwregister, precies de
faalvorm waar de AVG en het FDS beide tegen duwen.
@@ -0,0 +1,42 @@
# ADR-0005: Verwerkingenlog via event-emissie, in lijn met LDV
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle, FG (geconsulteerd)
- **Vervangt / vervangen door:**
## Context
AVG art. 30 vereist een register van verwerkingsactiviteiten. De FDS-bouwsteen Logboek
Dataverwerkingen (LDV) wijst naar een gestandaardiseerd verwerkingslog dat de burger kan bevragen.
Database-CDC met Debezium legt *datawijzigingen* vast, en niet *verwerkingsgebeurtenissen met
doelbinding*. Het is dus geen verwerkingenlog.
## Besluit
Elke registeradapter stuurt een **verwerkingsactiviteit-event** naar een eigen Redpanda-topic, via een
Verwerking Port en een LDV Emitter. Het event bevat: subjectcategorie, register, velden, doel en
doelbinding, grondslag, bevragende rol, en tijdstempel. **Nooit de opgehaalde waarden.**
Een projectie maakt het log bevraagbaar. De emissie is asynchroon, maar niet over te slaan: de adapter
die de Register Port vervult, is dezelfde code die het event uitstuurt.
## Gevolgen
**Positief:** het spoor voor art. 30 en LDV ontstaat als neveneffect van de bevraging, dus het kan niet
uit de pas lopen met de werkelijkheid. Het is af te stemmen tegen de tweezijdige logs van FSC
(ADR-0002). Het is onderscheidend in een tender.
**Negatief en kosten:** een topic en een projectie om te exploiteren. Het ontsluiten van het log naar
de burger valt buiten de huidige scope; wij produceren het log. Het eventschema vraagt governance.
**Vervolgwerk:** definieer het schema van het verwerkingsevent. Bouw de bevraagbare projectie. Sluit
aan op de LDV-standaard zodra die volwassen wordt; dit is een upstream-kandidaat.
## Overwogen alternatieven
- **Debezium-CDC hergebruiken als log** — afgewezen: dat legt datawijzigingen vast, en geen verwerking
met doelbinding. Verkeerde semantiek.
- **Synchroon loggen in het aanroeppad** — afgewezen: dat koppelt de latency van de bevraging aan het
log. Asynchroon maar niet over te slaan geeft zowel snelheid als garantie.
@@ -0,0 +1,68 @@
# ADR-0006: Modulegrens en hergebruikstrategie voor de governed-access spine
- **Status:** accepted
- **Datum:** 2026-06-13
- **Deciders:** Lab Circle (Lead Link, Build, Upstream Liaison)
- **Vervangt / vervangen door:**
## Context
De compliance-spine uit slice 1 bestaat uit de PDP-controle (ADR-0003), gegoverneerd uitgaand verkeer
via FSC (ADR-0002), emissie van het verwerkingenlog (ADR-0005), en de begrensde cache (ADR-0004),
allemaal achter ports (ADR-0001). Die spine is mogelijk breder herbruikbaar dan alleen in de
referentie-applicatie.
Er spelen twee hergebruikvragen: welke verpakkingsvorm kiezen wij, en hoe verhoudt de spine zich tot
andere omgevingen zoals het OpenMetadata-datagovernanceproject?
Twee verduidelijkingen bepalen het besluit.
1. **OpenMetadata is geen afnemer.** In het datagovernanceproject is het de catalogus- en
lineage-laag over (synthetische) data. Het bevraagt geen BRP of NHR. FSC of de begrensde cache
daarin inbouwen zou zinloos zijn. De juiste aansluiting is **integratie van de output van de
spine**, en niet het inbouwen van de spine.
2. **FSC en de begrensde cache zijn zaken die alleen een afnemer aangaan.** "Maak het herbruikbaar"
mag deze niet uitsmeren over componenten die geen registerdata bevragen.
Nu al een taalonafhankelijke gateway bouwen — vóórdat er een tweede, niet-.NET afnemer bestaat — zou
de valkuil van speculatieve architectuur herhalen, die wij voor de capability-laag al hebben
afgewezen.
## Besluit
Wij nemen een **vraaggestuurde reeks van vier stappen** aan. Elke stap hangt af van echte behoefte, en
niet van verwachte behoefte.
| Stap | Wat | Wanneer |
|---|---|---|
| 1 | **In-process bewijzen.** Bouw de spine als gewone componenten achter ports, binnen de .NET register-applicatie. Nog geen extractie. Doel: de compliance-invarianten één keer echt valideren. | Slice 1 |
| 2 | **Extraheren als .NET-module.** Zodra een tweede .NET-afnemer in zicht is, haal de spine eruit als een geversioneerde .NET-library of SDK. Dit is de ACL-template-extractie die het charter al plant. Herbruikbaar voor .NET-afnemers, en dat is genoeg voor register-reference en zijn broertjes. | Slice 3 |
| 3 | **De feed LDV naar OpenMetadata aansluiten.** Route verwerkingsevents uit de LDV-emitter naar OpenMetadata als access- en usage-metadata bij het geclassificeerde asset: wie las welk persoonsgegevensveld, met welk doel, hoe vaak. Optioneel laten classificatietags uit OpenMetadata terugstromen om veldminimalisatie in de ACL aan te sturen. Dit is de concrete brug tussen beide anchor-projecten: integratie, geen inbouw. | Na stap 2 |
| 4 | **Alleen op verzoek een taalonafhankelijke gateway bouwen.** Heeft een echte niet-.NET afnemer gegoverneerde registertoegang nodig, verpak de spine dan als zelfstandige sidecar of proxy met een dunne lokale API, met PDP, FSC-egress en LDV erachter. Niet eerder. | Op verzoek |
## Gevolgen
**Positief:** eigen software blijft minimaal. Hergebruik volgt op validatie in plaats van eraan vooraf
te gaan. Beide anchor-projecten krijgen een concreet, benoemd integratiepunt (stap 3). Zaken die
alleen een afnemer aangaan, blijven ingesloten.
**Negatief en kosten:** de .NET-module uit stap 2 dient geen niet-.NET afnemers. Dat aanvaarden wij,
omdat stap 4 dat geval dekt zodra het echt is. Stap 3 vraagt een afgesproken schema voor het
verwerkingsevent, stabiel genoeg voor OpenMetadata om te consumeren.
**Vervolgwerk:**
1. Neem stap 3 als expliciet integratiepunt op in beide projectpagina's in het Innovation Lab-repo:
`projects/open-register-fd/README.md` en `projects/openmetadata/README.md`.
2. Herzie de trigger van stap 4 bij elke portfolio-review. Bouw niet vooruit.
3. Regel governance op het schema van het verwerkingsevent; dat is een gedeelde afhankelijkheid van
stap 1 en stap 3.
## Overwogen alternatieven
- **De taalonafhankelijke gateway vooraf bouwen** — afgewezen: speculatieve architectuur voordat er een
tweede afnemer bestaat. De latency en de operationele kosten zijn niet te rechtvaardigen.
- **De spine in OpenMetadata inbouwen** — afgewezen: OpenMetadata is geen afnemer. Dit is een
categoriefout.
- **De spine permanent in-process houden, zonder extractie** — afgewezen: dat geeft het hergebruik
tussen projecten en applicaties weg, en dat is een kerndoel van de Open Register-inzet.
+27
View File
@@ -0,0 +1,27 @@
# ADR-NNNN: <titel>
- **Status:** proposed
- **Datum:** JJJJ-MM-DD
- **Deciders:** <rollen>
- **Vervangt / vervangen door:**
## Context
<De krachten die spelen: het probleem, de beperkingen, de FDS- en AVG-drijfveren. Waarom er nu een
besluit nodig is.>
## Besluit
<De keuze, eenvoudig gesteld.>
## Gevolgen
**Positief:** <wat dit oplevert>
**Negatief en kosten:** <wat het kost, en wat wij aanvaarden>
**Vervolgwerk:** <welk werk dit oproept>
## Overwogen alternatieven
<De afgewezen opties, en waarom.>
+127
View File
@@ -0,0 +1,127 @@
# C4-componentview — register-applicatie en capability-laag
> Niveau 3, de componentview. Deze view zoomt in op de container van de .NET register-applicatie uit
> het L2-containerdiagram. Zij verbindt het geheel op componentniveau — domein, ports, adapters en de
> FDS-capability-componenten — en toont waar elk onderdeel externe tooling raakt.
>
> De hexagonale structuur is expliciet: het domein hangt alleen af van **ports** (interfaces). Elke
> concrete capability is een **adapter** die aan een port is gebonden.
>
> De containerview (L2), de blauwdruk en de privacy-datastroomviews staan in het Innovation Lab-repo,
> `Respellion/innovation-lab`, onder `projects/open-register-fd/`.
```mermaid
C4Component
title Componentview — register-applicatie (.NET) en de FDS-capability-laag
Person(user, "Behandelaar", "Behandelt zaken")
Container(spa, "Frontend", "Angular + NL Design System", "Zaakinterface")
Container_Boundary(app, "Register-applicatie (.NET, hexagonaal)") {
Component(api, "API / application services", ".NET", "Orkestreert use cases; verklaart doelbinding per vraag")
Component(domain, "Domeinmodel", ".NET / DDD", "Ubiquitous language; geen registervocabulaire")
Component(portReg, "Register Port", "interface", "De vraag van het domein: Personen / Organisaties")
Component(portPol, "Authorisation Port", "interface", "mag-deze-verwerking-doorgaan?")
Component(portLog, "Verwerking Port", "interface", "leg de verwerkingsgebeurtenis vast")
Component(portTm, "Terugmelding Port", "interface", "meld een vermoedelijke fout")
Component(portCache, "Cache Port", "interface", "doelgebonden lezen, schrijven en verwijderen")
Component(aclBrp, "BRP-adapter", ".NET", "Vertaalt domein<->BRP; minimale velden")
Component(aclKvk, "NHR/KVK-adapter", ".NET", "Vertaalt domein<->NHR; UBO-bewust")
Component(pdpClient, "PDP Client", ".NET -> OPA", "Roept de policy engine; geeft doel en grondslag mee")
Component(ldvEmit, "LDV Emitter", ".NET", "Bouwt het verwerkingsevent; publiceert naar Redpanda")
Component(fscClient, "FSC Client", ".NET", "Stuurt contractuele aanroepen via de outway")
Component(cacheMgr, "Cache Manager", ".NET", "TTL en verwijderen op subjectsleutel")
Component(tmHandler, "Terugmelding Handler", ".NET -> Flowable", "Start het terugmeldproces")
Component(procClient, "Process Client", ".NET -> Flowable", "Uitvoering van BPMN en DMN")
}
System_Ext(opa, "OPA (PDP)", "Policies geversioneerd in Gitea")
System_Ext(fsc, "FSC Outway", "EUPL-referentie-implementatie")
System_Ext(flowable, "Flowable", "BPMN + DMN")
ContainerDb_Ext(cache, "Begrensde cache", "PostgreSQL")
System_Ext(redpanda, "Redpanda", "LDV-topic + CDC")
System_Ext(brp, "BRP", "via FSC inway")
System_Ext(kvk, "NHR / KVK", "via FSC inway")
System_Ext(kanidm, "Kanidm", "OIDC")
Rel(user, spa, "Gebruikt")
Rel(spa, api, "REST/JSON")
Rel(kanidm, api, "OIDC", "authenticatie")
Rel(api, domain, "Roept aan")
Rel(api, portPol, "Controleert vóór de bevraging")
Rel(api, portReg, "Vraagt data")
Rel(api, portTm, "Dient melding in")
Rel(api, procClient, "Voert proces uit")
Rel(portPol, pdpClient, "gebonden aan")
Rel(pdpClient, opa, "besluitverzoek")
Rel(portReg, aclBrp, "gebonden aan")
Rel(portReg, aclKvk, "gebonden aan")
Rel(aclBrp, fscClient, "via")
Rel(aclKvk, fscClient, "via")
Rel(aclBrp, portLog, "stuurt event")
Rel(aclKvk, portLog, "stuurt event")
Rel(aclBrp, portCache, "leest en schrijft")
Rel(aclKvk, portCache, "leest en schrijft")
Rel(fscClient, fsc, "contractuele aanroep")
Rel(fsc, brp, "mTLS + contract")
Rel(fsc, kvk, "mTLS + contract")
Rel(portLog, ldvEmit, "gebonden aan")
Rel(ldvEmit, redpanda, "publiceert")
Rel(portCache, cacheMgr, "gebonden aan")
Rel(cacheMgr, cache, "slaat op")
Rel(portTm, tmHandler, "gebonden aan")
Rel(tmHandler, flowable, "start proces")
Rel(procClient, flowable, "voert uit")
```
## Hoe je dit leest
1. **De ports zijn de naad.** Het domein en de application services hangen af van de vijf interfaces,
en nooit van adapters. FSC wisselen voor DSP, of OPA voor de latere FTV-client, verandert een
adapter — geen port, en niet het domein. Dit is de clock-speed boundary, concreet gemaakt.
2. **De compliance-componenten zijn adapters, geen domeinlogica.** De PDP-client, de LDV-emitter, de
FSC-client en de cache manager staan allemaal aan de adapterzijde. Een bevraging kan er fysiek niet
langs, omdat de adapter die de Register Port vervult dezelfde code is die het LDV-event uitstuurt
en via FSC routeert.
3. **Slechts twee componenten raken de registers**: de BRP-adapter en de NHR/KVK-adapter. Beide
bereiken ze uitsluitend via de FSC-client. Er is geen vierde pad.
## Componenten tegenover verplichtingen
| Component | Omvang eigen bouw | Verplichting die het afdekt |
|---|---|---|
| Domeinmodel | het product | correctheid van de bedrijfsregels |
| BRP- en NHR-adapters | dun | dataminimalisatie: vertalen en velden versmallen |
| PDP Client | klein | handhaven van grondslag en doelbinding |
| LDV Emitter | klein | verwerkingenlog (AVG art. 30 en LDV) |
| FSC Client | klein | geautoriseerde, gelogde connectiviteit |
| Cache Manager | klein | grenzen aan bewaring, en verwijdering |
| Terugmelding Handler | klein | de terugmeldplicht van de afnemer |
---
## Aanvullende views die voor engineers waarde hebben
De diagrammen tot hier verklaren *structuur* en *compliance-intentie*. Engineers die dit bouwen,
hebben er nog een aantal nodig. Wij tekenen geen view voordat er iets echt is om te beschrijven, dus
elke regel noemt de trigger.
| # | View | Wat het toevoegt | Trigger |
|---|---|---|---|
| 1 | **Deploymentview** (C4 deployment, topologie) | Waar elke container op De Werf draait: k3s-namespaces, welke services sidecar zijn en welke een eigen pod (is OPA een sidecar of centraal? waar eindigt de FSC outway?), netwerkpolicies tussen de vlakken van de vertrouwensgrens, en beheer van secrets en mTLS-certificaten voor FSC. Hier worden de privacy*grenzen* echte firewall- en netwerkregels. | Vóór de eerste deploy met meerdere services. **Hoogste waarde als volgende.** |
| 2 | **Sequences voor de niet-gelukkige paden** | Wij hebben het gelukkige pad. Engineers hebben de lastige nodig: PDP-*deny* midden in een transactie, een verlopen of ingetrokken FSC-contract, een register-timeout terwijl er een verouderde cache-entry ligt, en een gedeeltelijk NHR-antwoord waarbij een UBO-veld is achtergehouden. Dit bepaalt de foutafhandeling, en hier verstoppen de compliance-randgevallen zich. | Direct na slice 1. |
| 3 | **Domeinmodel en ERD** | De bounded contexts en aggregates in het domein, plus het cacheschema: welke persoonsgegevens blijven staan, op welke sleutel, en met welke purge-kolom. Dit is tegelijk het artefact dat de FG beoordeelt voor bewaartermijnen. | Zodra het domein in slice 1 stabiliseert. |
| 4 | **Dataclassificatie- en catalogusview** | Elk veld dat een grens kruist, getagd — persoonsgegeven? bijzondere categorie? UBO-beperkt? — en gemapt op zijn classificatie in OpenMetadata. Dit stuurt de GDPR-scrubbingregels en de lineage-tags. | Beter *uit* OpenMetadata gegenereerd zodra die gevuld is, dan met de hand getekend. |
| 5 | **Toestandsdiagram: levensloop van een cache-entry** | `fetched``valid` (binnen TTL) → `stale``purged` (TTL verstreken \| zaak gesloten \| verwijderingsverzoek). Klein, maar het pint de bewaarsemantiek vast die "begrensde cache" nu alleen in prose beschrijft. | Samen met ADR-0004-vervolgwerk. |
| 6 | **BPMN-view: de terugmelding-workflow** | Het Flowable-proces zelf: ingediend → verstuurd naar bronhouder → bevestigd → opgelost of afgewezen. Dit is uitvoerbaar BPMN, dus het diagram en de implementatie zijn hetzelfde artefact. | Wanneer de terugmelding-slice start. |
| 7 | **Threat model en vertrouwensgrensview** (STRIDE-stijl) | Dreigingen over de vertrouwensgrens leggen: tokendiefstal, cache poisoning, replay tegen FSC, policy bypass, en manipulatie van logs. Past natuurlijk bij de FSC-zoom, en is het anker van het securitygesprek. | Vóór het verwerken van echte persoonsgegevens. |
| 8 | **CI/CD- en policy-promotieview** | Hoe OPA-policies en BPMN/DMN-modellen van een pull request naar draaiende configuratie gaan. "Toegangsbeheer is configuratie in Gitea" geldt alleen als er een pijplijn is die review en promotie handhaaft. | Samen met het vervolgwerk uit ADR-0003. |
**Voorstel voor de volgende twee.** De **deploymentview**, omdat die de privacygrenzen omzet in
handhaafbare netwerkpolicy. En de **sequences voor de niet-gelukkige paden**, omdat compliance daar
werkelijk breekt.
+103
View File
@@ -0,0 +1,103 @@
# POC-voorstel — slice 1: walking skeleton (één register, gegoverneerde bevraging)
> Klaar om in een `poc-voorstel`-issue te plakken, met de labels `build` en `poc`. Dit is het bouwbare
> eerste increment dat de architectuurdocumenten beschrijven. Het bewijst met opzet de
> *compliance-spine* end-to-end op de dunst mogelijke functionaliteit.
## Probleem en strategische vraag
Kunnen wij een registerbevraging demonstreren die *structureel* gegoverneerd is — onmogelijk uit te
voeren zonder gehandhaafde grondslag en een automatische regel in het verwerkingenlog — op onze
soevereine stack?
Dit is de geloofwaardigheidstoets achter de hele Open Register-inzet (slice 1 van het charter) en
achter de FDS gap-analyse.
## Hypothese
Wij verwachten dat het doorverbinden van één registerbevraging door de volledige capability-spine —
Register Port → ACL-adapter → PDP-controle → FSC-aanroep → LDV-emissie → begrensde cache — de claim
"compliance is structureel" bewijst.
Wij weten dat wij het goed hebben als een geautomatiseerde test aantoont dat een bevraging **niet** kan
voltooien als de PDP weigert, en **altijd** een LDV-event oplevert als de PDP toestaat.
## Scope ter grootte van één blok
**Wel in scope**
| Onderdeel | Wat |
|---|---|
| Register | **NHR/KVK**, basisgegevens over onderneming en bestuurder. Gekozen boven BRP; zie de slotnotitie. |
| Use case | Geef bij een KVK-nummer de geregistreerde organisatie terug aan het domein, voor één verklaard doel. |
| Ports | De vijf ports als interface. Concrete adapters: NHR-ACL, PDP-client (OPA), FSC-client met sandbox- of test-outway, LDV-emitter (Redpanda-topic), en cache manager (PostgreSQL met TTL). |
| Policy | OPA draait met één handgeschreven voorbeeldpolicy in Gitea: één allow-regel en één deny-geval. |
| Log | Verwerkingsevent-schema v0 plus een minimale bevraagbare projectie; een tabelweergave is genoeg. |
| Tests | Tests die de twee compliance-invarianten vastleggen: deny blokkeert, allow logt. |
**Niet in scope** — even belangrijk om op te schrijven.
1. Afgewerkte interface of NL Design System-schermen, verder dan een dev-harness.
2. BRP en paden met veel persoonsgegevens. Die gaan naar slice 2, met een door de FG beoordeelde
policy.
3. UBO-data. Het regime van beperkte toegankelijkheid valt buiten deze slice.
4. De terugmelding-workflow (latere slice), DCAT-export, en Superset-dashboards.
5. Echte register-endpoints. Alleen sandbox en stubs.
## Definition of Done
- [ ] Een bevraging op KVK-nummer geeft een domein-`Organisatie` terug via de NHR-ACL-adapter, zonder
registervocabulaire in het domein (ADR-0001).
- [ ] De aanroep loopt via de FSC-client naar een sandbox-outway, en niet via een ruwe HTTP-client
(ADR-0002).
- [ ] Er vindt geen bevraging plaats tenzij de PDP allow teruggeeft voor de combinatie rol, doel en
grondslag (ADR-0003).
- [ ] Elke toegestane bevraging stuurt precies één verwerkingsevent naar Redpanda, bevraagbaar in de
projectie, zonder opgehaalde waarden (ADR-0005).
- [ ] Cache-entries dragen een TTL en een subjectsleutel; een purge-aanroep verwijdert ze (ADR-0004).
- [ ] **De tests op de compliance-invarianten slagen in CI:** (a) PDP-deny betekent geen FSC-aanroep;
(b) PDP-allow betekent precies één LDV-event; (c) te ruim gevraagde velden bereiken het domein
nooit.
- [ ] Het geheel draait lokaal uit een gedocumenteerd `compose`- of k3s-manifest met stubs, zonder
echte registertoegang.
- [ ] ADR-0001 tot en met ADR-0005 zijn vanuit de code gelinkt. Eén nieuwe ADR als er in slice 1 een
besluit ontstaat.
## Acceptatiedemo (bewijs voor de week-3-toets)
Live: een geslaagde bevraging plus de bijbehorende LDV-regel. Zet daarna de policy op deny en toon
dezelfde bevraging geweigerd, zonder registeraanroep en zonder data.
Dat contrast *is* de demo.
## Ontvangende Delivery Circle (voorlopig)
De register-reference Delivery Circle. De Handoff-ontvanger krijgt bij de kickoff een naam.
Waarschijnlijke adoptie: de capability-spine wordt het herbruikbare substraat voor de
register-reference-applicatie.
## Upstream-kandidaten
| Project | Wat wij kunnen bijdragen |
|---|---|
| fsc-nlx | Ergonomie van de sandbox en testomgeving, plus documentatie |
| OPA | Policy-patronen voor het modelleren van Nederlandse grondslagen |
| OpenMetadata | Later een DCAT-AP-NL exporter; dit verbindt het OpenMetadata-project |
## AVG- en soevereiniteitsoverwegingen
Alleen NHR-basisgegevens, over onderneming en bestuurder, en in slice 1 **gestubd**. Er worden geen
echte persoonsgegevens verwerkt.
Een FG-review is een voorwaarde voor slice 2, met echte data en BRP. Alle componenten draaien
zelfgehost op De Werf; OPA-policies en BPMN staan in Gitea.
## Slotnotitie: waarom NHR vóór BRP voor het skeleton
Beide registers bevatten persoonsgegevens, dus geen van beide is "gratis". NHR-basisgegevens over
onderneming en bestuurder zijn echter minder gevoelig dan BRP-gegevens over inwoners, en er is een
duidelijker verhaal rond een publieke sandbox.
Zo bewijst slice 1 het *mechanisme*, voordat slice 2 BRP oppakt onder een door de FG beoordeelde
policy. UBO-data blijft buiten scope tot het toegangsregime is gemodelleerd.
+169 -5
View File
@@ -5,6 +5,138 @@ copy-pasteable walkthrough against a local `make up` stack.
---
## S-26/#162 — the werkbak refreshes itself (ADR-0032)
**Outcome:** a registration that reaches beoordeling while a behandelaar already has the werkbak open
**appears on its own** — no reload. The page re-reads `GET /behandel/werkbak` every 5 seconds; a
background refresh swaps the rows in without flashing the loading state, and a transient failure no
longer strands the view on its error message until someone reloads.
```bash
# 1. Two windows. Left: the behandel werkbak, already open and idle.
python3 infra/keycloak/check_realms.py otp # a code, valid right now
open http://localhost:8142 # merel-behandelaar / test123 + that code
#
# 2. Right: submit a registration and supply its documents (this is what routes it to Beoordelen).
open http://localhost:8140 # jan-burger / test123 → indienen → upload a PDF
#
# 3. Watch the left window. Within ~5 seconds the new reference appears in the werkbak — the page was
# never reloaded and never left the werkbak.
#
# 4. Automated, end to end: the happy path now waits for the werkbak row WITHOUT reloading, so the
# absence of the reload IS the assertion.
make verify-e2e # → registration.spec: "… → behandelaar goedkeurt → public INGESCHREVEN"
#
# 5. Component level (background refresh, failure recovery, teardown):
pnpm nx test behandel # → "picks up a newly submitted registration without a reload" (+3 guards)
```
**The path:** unchanged — portal → BFF `GET /behandel/werkbak` → domain `Werkbak` → Flowable. Only the
page's cadence is new: `interval(WERKBAK_REFRESH_MS)` scoped to the page with `takeUntilDestroyed()`.
**Not push:** nothing notifies the BFF either, so SSE/WebSockets would poll the domain inside the BFF
for the same freshness plus connection state — see ADR-0032 for the trade-off and the upgrade path.
---
## S-19a — approval writes the register record to Objecten (#149, ADR-0028)
**Outcome:** approving a registration no longer only moves the ZGW zaak to its eindstatus — it also
writes the canonical **register record** into the **Objecten** API. OpenZaak keeps the process,
Objecten holds the register. The write goes through the ACL (§8.1) and is **idempotent**: replaying an
approval updates the existing object instead of creating a second one.
```bash
# 1. Bring the stack up (Objecten, Objecttypen and the RegisterRecord objecttype come with it).
make up
#
# 2. End-to-end: the walking-skeleton e2e submits, approves via the behandel portal, and then
# asserts Objecten holds exactly one RegisterRecord for *that* registration:
make verify-e2e # → "DigiD submit → … → behandelaar goedkeurt → public INGESCHREVEN"
#
# 3. The ACL integration test proves the same writes against a live Objecten (upsert stays one object):
make verify-acl # → "Writes a register record and updates it in place on a second write"
#
# 4. See it for yourself — every register record currently in Objecten:
curl -s -H 'Authorization: Token 1234567890abcdef1234567890abcdef12345678' \
-H 'Accept-Crs: EPSG:4326' \
'http://localhost:8021/api/v2/objects' | python3 -m json.tool
```
Each object's `record.data` carries exactly `id`, `status`, `reference` — the schema forbids anything
else (ADR-0027), so no personal data can reach the world-readable register even by mistake.
**The path:** behandel portal → BFF → domain `BeoordeelRegistratie` → ACL `POST /statussen` → ZGW
`resultaten` + `statussen` (the process), **then** ACL → Objecten `POST`/`PATCH /api/v2/objects` (the
register). The objecttype URL is resolved by name from Objecttypen on first use, so nothing seed-time
is pinned in config (ADR-0028, same reasoning as ADR-0021).
**Not yet:** the public register still reads the NRC-derived projection — re-sourcing it from Objecten
is S-19b (#150).
---
## S-18c — RegisterRecord objecttype defined + registered (#141, ADR-0027)
**Outcome:** a **RegisterRecord** objecttype with a **published** JSON schema is registered in the
Objecttypen API at startup. The schema is public-safe by construction — `id`, `status`, `reference`
only, mirroring the BFF's `OpenbaarEntry` (no `bsn`/`naam`), `dataClassification: open`. This is the
schema S-19 writes register records against on approval. A `registerrecord-init` one-shot creates it
over the API once Objecttypen is healthy (the Objecttypen `setup_configuration` has no objecttype
step), idempotently.
```bash
make up
# The RegisterRecord objecttype exists with a published version:
curl -s -H "Authorization: Token 0123456789abcdef0123456789abcdef01234567" \
"http://localhost:8020/api/v2/objecttypes" | python3 -c \
'import sys,json; o=[x for x in json.load(sys.stdin)["results"] if x["name"]=="RegisterRecord"][0]; print(o["name"], o["dataClassification"], o["versions"])'
# → RegisterRecord open ['http://.../objecttypes/<uuid>/versions/1']
#
# Automated (a CI verify-stack step): asserts the objecttype exists, has a published version, and
# that version's schema carries id/status/reference.
make verify-registerrecord # → OK — RegisterRecord v1 published, fields=['id', 'reference', 'status']
```
**The path:** `infra/objecttypen-registerrecord/registerrecord.schema.json` (the reviewed public-safe
contract) + `register.py` are streamed into an external config volume by `infra/seed-config.sh
registerrecord` (bind-mounted locally); the `registerrecord-init` one-shot POSTs the objecttype + a
published version. Re-running is a no-op. S-19 (#20) writes records against this schema in Objecten.
---
## S-18b — Objecten API up in compose, wired to Objecttypen (#140)
**Outcome:** the upstream Maykin **Objecten API** runs in the stack — own **PostGIS** DB + redis,
config seeded like the other CG modules (`objecten-init` runs `setup_configuration` from the
`rr-objecten-config` volume: migrate + provision a dev **static API token** + register the
**Objecttypen API** (S-18a) as a trusted service), a health-checked `objecten` web on host `:8021`.
An object can now reference its objecttype; the ACL writes register records here on approval (S-19).
```bash
make up
# 1. The API is up; the seeded token authenticates (401 without, 200 with):
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8021/api/v2/objects # 401
curl -s -o /dev/null -w "%{http_code}\n" -H "Authorization: Token 1234567890abcdef1234567890abcdef12345678" \
http://localhost:8021/api/v2/objects # 200
#
# 2. It trusts Objecttypen — the seeded zgw_consumers service points at the Objecttypen API:
docker exec infra-objecten-1 python src/manage.py shell -c \
"from zgw_consumers.models import Service; print(*[(s.slug,s.api_root) for s in Service.objects.all()])"
# → ('objecttypen', 'http://objecttypen:8000/api/v2/')
#
# 3. Automated (a CI verify-stack step): asserts unauth 401 + token 200, against the running stack.
make verify-objecten # → OK — no-auth 401, token 200
```
**The path:** verbatim upstream image (`maykinmedia/objects-api`, pinned 3.4.0) + the same seed
pattern as S-18a — `infra/seed-config.sh objecten` streams `data.yaml` into an external config
volume, `objecten-init` (RUN_SETUP_CONFIG) applies it. Its `zgw_consumers` step registers Objecttypen
(`api_type: orc`, api-key auth with the S-18a dev token). The RegisterRecord objecttype (S-18c) and
the ACL write path (S-19) build on this.
---
## S-18a — Objecttypen API up in compose (#139)
**Outcome:** the upstream Maykin **Objecttypen API** runs in the stack — own Postgres + redis, config
@@ -42,7 +174,8 @@ zaaktype cache). Store is in-memory: an edit reverts to the configured env on re
```bash
make up
# 1. Log in as bram-beheerder / test123 → "Default-fill" tab → change a value → Opslaan.
# 1. Log in as bram-beheerder / test123 + OTP (`python3 infra/keycloak/check_realms.py otp`)
# → "Default-fill" tab → change a value → Opslaan.
open http://localhost:8143/default-fill
#
# 2. Automated: the ACL uses the current default-fill per zaak (unit) and the endpoints are behind the
@@ -63,7 +196,8 @@ directly (ADR-0025); managing the default-fill config (S-15b) and MFA (S-15c) co
```bash
make up
# 1. Log in as bram-beheerder / test123 → the catalogus lists the published zaaktypen.
# 1. Log in as bram-beheerder / test123 + OTP (`python3 infra/keycloak/check_realms.py otp`)
# → the catalogus lists the published zaaktypen.
open http://localhost:8143
#
# 2. Automated (a CI verify-stack e2e): a beheerder logs in and sees BIG-REGISTRATIE.
@@ -206,7 +340,8 @@ make verify-local # → "OK — a fresh local stack completed the flow with
# 3. Or by hand in the browser: log in at http://localhost:8140 (jan-burger / test123), submit +
# upload a PDF, then approve it in the werkbak at http://localhost:8142 (merel-behandelaar /
# test123); it shows as INGESCHREVEN in the openbaar register at http://localhost:8141.
# test123 + OTP, see S-15c); it shows as INGESCHREVEN in the openbaar register at
# http://localhost:8141.
```
> The zaaktype is discovered by the ACL itself since S-27 (below); `local-seed`'s `acl.env` now
@@ -254,7 +389,7 @@ make verify-e2e # → login as jan-burger → submit → "ontvangen" co
open http://localhost:8140
```
> The portal is served same-origin with the BFF (nginx proxies `/self-service` + `/openbaar`), so no
> The portal is served same-origin with the BFF (Caddy proxies `/self-service` + `/openbaar`), so no
> CORS; the OIDC authority comes from `/config.json` at runtime. See `docs/frontend-decisions.md`.
---
@@ -491,7 +626,7 @@ or **afwijzen** — which also completes the Beoordelen task so the process adva
```text
# 1. Open the behandel portal and log in as a behandelaar (medewerker realm):
# http://localhost:8142/ → merel-behandelaar / test123
# http://localhost:8142/ → merel-behandelaar / test123 + OTP
#
# 2. The werkbak lists the registrations awaiting beoordeling (referentie / bsn / status).
# Find the reference from the submit confirmation and click "Goedkeuren" on that row.
@@ -714,3 +849,32 @@ make verify-domain # → "the timed-out registration's zaak was cancelled to
`POST /annuleringen` → ZGW `resultaten` + `statussen` (Geannuleerd); the aggregate then moves to
`Verlopen`. The ACL cancels the zaak **before** the aggregate is expired, so a failed ZGW call leaves the
job for redelivery rather than diverging the two (ADR-0019).
---
## S-15c — MFA on the medewerker realm (#132, ADR-0031)
**Outcome:** staff logins (behandel + beheer portals) need a **second factor**. The medewerker realm
seeds every medewerker with a TOTP credential, so Keycloak's conditional-OTP step challenges them in
both the browser flow and the direct grant; a password alone no longer yields a token. `CONFIGURE_TOTP`
is a default required action, so a medewerker added later must enrol first. Citizen realms (digid,
eherkenning, eidas) are unchanged — they mock brokers that carry their own assurance.
```bash
# 1. Manual: log in to the behandel portal. After username + password Keycloak asks for a code.
python3 infra/keycloak/check_realms.py otp # a valid code, right now
open http://localhost:8142 # merel-behandelaar / test123 + that code
#
# 2. Automated: the realm smoke check asserts the password alone is REFUSED, then that
# password + TOTP succeeds and still carries the behandelaar role:
make keycloak-smoke # → "medewerker merel-behandelaar password-only login refused [OK]"
#
# 3. End-to-end: every staff login in the e2e goes through the OTP prompt (loginMedewerker):
make verify-e2e # → registration.spec (behandelaar approves), catalogus.spec, default-fill.spec
```
**The path:** the seeded `otp` credential in `infra/keycloak/realms/medewerker-realm.json` activates
Keycloak's stock conditional-OTP subflow — no custom browser flow. The fixture secret is shared and
committed on purpose so the checks can compute codes; a real deployment enrols per-user authenticators
(ADR-0031).
+8 -6
View File
@@ -77,11 +77,13 @@ with the submit form (S-08c, #67); any deviation from NL DS will be recorded her
## Serving + e2e (S-08d, #68)
- **Served by nginx, same-origin as the BFF.** The compose `self-service` image serves the built app
- **Served by Caddy, same-origin as the BFF.** The compose `self-service` image serves the built app
and **reverse-proxies** `/self-service/*` + `/openbaar/*` to the `bff` service. Because the
api-client uses **relative URLs**, the browser calls the app's own origin → nginx forwards to the
BFF: **no CORS**, and the DigiD token (same-origin) is attached by the interceptor. nginx resolves
the BFF at request time (a `resolver` + variable `proxy_pass`) so it starts before the BFF is up.
api-client uses **relative URLs**, the browser calls the app's own origin → Caddy forwards to the
BFF: **no CORS**, and the DigiD token (same-origin) is attached by the interceptor. Caddy dials
the BFF per request through the system resolver, so it starts before the BFF is up, picks up its
restarts, and resolves the bare `bff` name on every engine — compose, podman and Kubernetes
(ADR-0034; the `Caddyfile` sits next to each app's `Dockerfile`).
- **Runtime config.** The app fetches `/config.json` before bootstrap (`main.ts`); `appConfig` is a
factory. The dev default (`public/config.json`) points at `localhost:8180`; the Docker image bakes
the compose value (`keycloak:8080`). One build, per-environment OIDC authority.
@@ -110,7 +112,7 @@ with the submit form (S-08c, #67); any deviation from NL DS will be recorded her
`angular-auth-oidc-client`, no interceptor, and no `config.json``main.ts` bootstraps `appConfig`
directly with just `provideHttpClient` + `provideRouter`. This is the deliberate contrast to
self-service and keeps the app trivially cacheable/CDN-able.
- **Same-origin via nginx, like self-service.** The compose `openbaar` image serves the built app and
- **Same-origin via Caddy, like self-service.** The compose `openbaar` image serves the built app and
reverse-proxies `/openbaar` to the BFF; the api-client's relative calls stay same-origin (no CORS).
Served on `:8141`, health-checked over IPv4 (`127.0.0.1`), no Keycloak dependency.
- **Public-safe by construction.** The portal only ever sees the BFF's `OpenbaarProjection.PublicView`
@@ -138,7 +140,7 @@ frontend work is the medewerker realm auth and the werkbak/decide page. Wiring r
**BFF remains the security boundary** (`behandelaar` policy, 401/403 on `/behandel/*`, ADR-0013);
the frontend role signal is for display/UX, and the werkbak page surfaces a load failure (e.g. a
403 for a non-behandelaar) rather than swallowing it.
- **Same-origin via nginx, like the other portals.** The compose `behandel` image serves the built
- **Same-origin via Caddy, like the other portals.** The compose `behandel` image serves the built
app and reverse-proxies `/behandel` to the BFF (relative calls, no CORS). Served on `:8142`,
health-checked over IPv4 (`127.0.0.1`), depends on Keycloak for the medewerker realm.
- **Werkbak = decide-and-refresh.** `WerkbakPage` loads `GET /behandel/werkbak` on open and renders a
+5
View File
@@ -9,8 +9,13 @@ should teach.
- **[Product Requirements](PRD.md)** — what we're building and why.
- **[ADR-0001: Loose coupling](architecture/adr-0001-loose-coupling.md)** — the
non-negotiable integration stance; the template for future ADRs.
- **[FDS architecture](architecture/fds/README.md)** — participating in the Federatief
Datastelsel as an afnemer: FDS ADR-0001…0006, the L3 component view, the slice-1 proposal.
In Dutch; the strategic framing lives in `Respellion/innovation-lab`.
- **[Working in Gitea](gitea-workflow.md)** — issues, milestones, branches, PRs.
- **[CI runbook](runbooks/ci.md)** — the pipeline and the `make ci` local gate.
- **[Kubernetes on Talos](runbooks/kubernetes-talos.md)** — the second deployment target:
one Helm chart, a single-node cluster, and the parts that bite (ADR-0033).
## Quickstart
+10 -2
View File
@@ -2,8 +2,10 @@
> **Status: active.** The workflow `.gitea/workflows/ci.yaml` runs on Gitea's
> hosted `ubuntu-latest` runner — no self-hosted runner required.
> **`make ci` is still the local gate** — it runs the exact same checks
> (the workflow calls the same `make` targets).
> **`make ci` is still the local gate** — it runs the same checks via the same
> `make` targets, with one exception: the `k8s` job's targets are not in `make ci`,
> because `helm` is optional for everyone not deploying to Kubernetes. Run
> `make k8s-lint k8s-drift` by hand after touching the chart or the compose file.
## The pipeline
@@ -16,6 +18,8 @@ and CI cannot drift:
| `lint` | `make lint``dotnet format … --verify-no-changes` | .NET 10 SDK |
| `build` | `make build``dotnet build … -c Release` | .NET 10 SDK |
| `unit` | `make unit``dotnet test … -c Release --filter "Category!=Integration"` | .NET 10 SDK |
| `frontend` | `make frontend` → Nx lint/test/build for the four portals | pnpm + Node |
| `k8s` | `make k8s-lint` (render + schema-check the Helm chart) → `make k8s-drift` (chart still describes the same stack as `infra/docker-compose.yml`) | pinned `helm` binary + `docker compose` |
| `mutation` | `make mutation``dotnet tool restore``dotnet stryker` (ACL); uploads the HTML report as an artifact | .NET 10 SDK |
| `verify-stack` | the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
@@ -27,6 +31,10 @@ and CI cannot drift:
> services by **container IP** (the runner can't reach published ports — see
> [gitea-actions-gotchas.md §5/§6](gitea-actions-gotchas.md)).
A second workflow, `.gitea/workflows/deploy.yaml`, deploys the stack to the Talos
cluster on the lab server when a PR is merged to `main` — see
[kubernetes-talos.md §9](kubernetes-talos.md) for its secrets and the SSH tunnel it needs.
All `uses:` references are absolute, tag-pinned URLs (`https://github.com/actions/checkout@v4`,
`https://github.com/actions/setup-dotnet@v4`) per CLAUDE.md §8.7 and §15 — Gitea
Actions resolves them from GitHub.
+44
View File
@@ -245,3 +245,47 @@ the verify-stack check table, and per-spec e2e results (`infra/playwright-summar
- Getting a report out of the e2e container: Playwright writes `playwright-report.json`
inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host
(capturing the test exit code first) so the summary step can read it.
---
## 9. `if: always()` does not survive the job being killed — bound the work itself
`if: always()` makes a step run when an *earlier step failed*. It does **not** help when
the job as a whole is stopped: the run's remaining steps are simply never dispatched.
That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17
and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e
shows a **0-second `failure`** stamped at that same instant:
```
14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success)
15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always()
16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always()
17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure()
18 failure 10:14:54 -> 10:14:54 Tear down ← if: always()
```
So the per-spec summary, the container-log dump and the teardown never ran, and the job
log — which also loses whatever the killed process had buffered — ended at a single `✘`
line. A job that dies takes its own post-mortem with it.
**Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}`
returns every step with `started_at`/`completed_at`; a row of identical zero-length
steps at the end means *killed*, not *silent*. (Job ids come from
`…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a
re-run hides the failed one, so keep the failing job id from the original report. Logs:
`…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.)
**Conventions that follow:**
- **Bound long-running work inside the tool**, where it can still report. Playwright's
`globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON
report and exits, so the summary and log-dump steps still get their turn. A
`timeout-minutes` on the job would reproduce the very failure above.
- **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`,
`click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs
the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the
element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`).
- Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same
ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before
concluding a job hung.
+36
View File
@@ -23,6 +23,9 @@ login per realm and asserts the identifying claim:
| eidas | pierre-dupont | `eidas_id` |
| medewerker | merel-behandelaar | role `behandelaar` |
The medewerker row also asserts that the password **alone** is refused — that realm
enforces MFA (below).
All test users / credentials are in [../synthetic-data.md](../synthetic-data.md).
## Notes
@@ -35,3 +38,36 @@ All test users / credentials are in [../synthetic-data.md](../synthetic-data.md)
- **Image** pinned to `quay.io/keycloak/keycloak:26.1`.
- Claims are injected by OIDC protocol mappers on `big-portal` (user attribute → token
claim); `medewerker` roles come through `realm_access.roles`.
## MFA on the medewerker realm (S-15c)
Staff logins (behandel + beheer portals) need a second factor; citizen/company realms
(digid, eherkenning, eidas) do not. Two halves in `medewerker-realm.json`:
- Every seeded medewerker carries a **TOTP credential** with the fixture secret
`BIGMEDEWERKEROTPSEED`, so Keycloak's built-in *conditional OTP* step fires on every
login — browser flow (an `#otp` prompt after the password) and direct grant (a `totp`
form field) alike.
- `CONFIGURE_TOTP` is a **default required action**, so any medewerker added later must
enrol an authenticator before the first login.
See [../architecture/adr-0031-mfa-on-the-medewerker-realm.md](../architecture/adr-0031-mfa-on-the-medewerker-realm.md).
### Getting a code
```bash
python3 infra/keycloak/check_realms.py otp # prints a valid 6-digit code right now
```
Or enrol a phone once: the secret in base32 is `IJEUOTKFIRCVORKSJNCVET2UKBJUKRKE`
(`otpauth://totp/medewerker?secret=IJEUOTKFIRCVORKSJNCVET2UKBJUKRKE`). The e2e computes its
own code in `tests/e2e/medewerker-login.ts`.
**A code is single-use.** Keycloak's `otpPolicyCodeReusable` defaults to false, so it refuses a
code it has already accepted — a second login as the same medewerker inside the same 30-second
window fails with `invalid_grant` / *Invalid user credentials*, even though the code is current.
Nothing to fix in the realm: wait for the next window, or spend the following counter, which is
what `nextUnusedCounter` in `tests/e2e/medewerker-login.ts` does for back-to-back specs.
**Fixture only.** A shared, committed secret is a demo convenience, never a production
posture — see the ADR's consequences.
+414
View File
@@ -0,0 +1,414 @@
# Deploying the stack to a single-node Talos cluster
The Helm chart in `infra/helm/big-reference` is a port of `infra/docker-compose.yml`
(ADR-0033). This runbook is the walkthrough that was actually used to bring the stack up
on a Talos VM under virt-manager on a laptop, including the parts that bite.
Compose remains the CI-canonical stack — `make verify`, the acceptance lane and the
Playwright e2e all still drive it. Kubernetes is a second deployment target.
## 0. What you need
On the laptop, four static binaries, all installable to `~/.local/bin` without root:
```bash
curl -sSLo ~/.local/bin/talosctl https://github.com/siderolabs/talos/releases/download/v1.14.0/talosctl-linux-amd64
curl -sSLo ~/.local/bin/kubectl https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl
curl -sSL https://get.helm.sh/helm-v3.16.4-linux-amd64.tar.gz | tar xz -O linux-amd64/helm > ~/.local/bin/helm
curl -sSL https://github.com/google/go-containerregistry/releases/download/v0.20.2/go-containerregistry_Linux_x86_64.tar.gz | tar xz -O crane > ~/.local/bin/crane
chmod +x ~/.local/bin/{talosctl,kubectl,helm,crane}
```
Match `talosctl` to the Talos ISO you booted (`talosctl version --insecure -n <ip>` reports
the server's tag). `crane` is what pushes images to a plain-HTTP registry without a
root-level Docker daemon change — see §2.
**VM sizing.** 6 vCPU / 10 GB RAM / 27 GB disk runs the whole stack with room to spare
(measured: ~4.4 GB used, 5.4 GB available with all 29 pods up). 4 GB is not enough. The
chart sets no resource requests or limits on purpose — on a single node the VM's RAM is the
only budget there is. Resize a stopped VM with:
```bash
virsh -c qemu:///system destroy talos # it's in maintenance mode; nothing is lost
virsh -c qemu:///system setmaxmem talos 10G --config
virsh -c qemu:///system setmem talos 10G --config
virsh -c qemu:///system setvcpus talos 6 --config --maximum
virsh -c qemu:///system setvcpus talos 6 --config
```
Two addresses matter throughout:
| Name | Meaning | Example |
|---|---|---|
| `TALOS_HOST` | the VM's IP — used by the browser, `talosctl` and `kubectl` | `192.168.122.33` |
| `K8S_REGISTRY` | `TALOS_HOST:30500` — the in-cluster registry (§2) | `192.168.122.33:30500` |
Find the VM's address with `virsh -c qemu:///system net-dhcp-leases default`.
## 1. Install Talos onto the VM
### The virt-manager trap
virt-manager treats the install ISO as one-shot: on the VM's **first shutdown** it ejects
the CD and rewrites the boot order to `hd`. A Talos VM booted from `metal-amd64.iso` runs
entirely in RAM, so the disk is still empty — the next start lands on
`Boot failed: not a bootable disk`. Put the ISO back before installing:
```bash
virsh -c qemu:///system change-media talos sda /path/to/metal-amd64.iso --config --insert
virt-xml -c qemu:///system talos --edit --boot cdrom,hd
virsh -c qemu:///system start talos
```
Wait for the maintenance-mode API, then confirm the install disk's device name — on virtio
it is `/dev/vda`, and Talos's default selector expects `/dev/sda`:
```bash
talosctl get disks --insecure -n <TALOS_HOST> -e <TALOS_HOST>
```
### Generate the machine config
Talos 1.14 moved several v1alpha1 fields into their own config documents. In particular
`machine.install` is now `UnattendedInstallConfig`, and patching the old field is rejected
with *"UnattendedInstallConfig config is incompatible with v1alpha1 config"*. Write
`patch.yaml` as a multi-document patch:
```yaml
machine:
certSANs:
- 192.168.122.33
registries:
mirrors:
# The in-cluster registry (§2) speaks plain HTTP.
"192.168.122.33:30500":
endpoints:
- http://192.168.122.33:30500
---
apiVersion: v1alpha1
kind: UnattendedInstallConfig
provisioning:
diskSelector:
match: disk.dev_path == "/dev/vda"
```
```bash
talosctl gen config big https://<TALOS_HOST>:6443 --output-dir ~/.talos/big --config-patch @patch.yaml
talosctl apply-config --insecure -n <TALOS_HOST> -e <TALOS_HOST> --file ~/.talos/big/controlplane.yaml
```
Talos installs to the disk and **kexecs straight into the installed system**, so the CD
boot order doesn't get in the way here. Then point the client at the node and bootstrap:
```bash
talosctl config merge ~/.talos/big/talosconfig
talosctl config endpoint <TALOS_HOST>
talosctl config node <TALOS_HOST>
talosctl bootstrap # wait for `talosctl version` to answer first
talosctl kubeconfig -f ~/.kube/config
```
A single-node cluster must run workloads on the control plane, or CoreDNS never schedules:
```bash
kubectl taint node --all node-role.kubernetes.io/control-plane-
```
Finally, make a VM restart boot the installed system rather than the ISO (takes effect at
the next full power cycle):
```bash
virsh -c qemu:///system change-media talos sda --eject --config
virt-xml -c qemu:///system talos --edit --boot hd
```
## 2. A registry the node can pull from
Talos has no Docker daemon and no way to side-load an image, so this repo's images have to
come from a registry. The registry runs **inside the cluster**, published on NodePort
30500 (`infra/helm/registry.yaml`):
```bash
make k8s-registry
```
Why in-cluster rather than on the laptop: a laptop-side registry needs an inbound port
opened on firewalld's `libvirt` zone (`sudo firewall-cmd --zone=libvirt --add-port=5000/tcp`),
which needs root. Pushing from the laptop *to* the node is outbound and always allowed, and
the node pulls from its own NodePort. If you do open that port, put a registry on the
laptop instead and point `K8S_REGISTRY` at `<laptop-ip>:5000` — the mirror patch in §1 has
an entry ready for it.
Its storage is `emptyDir`, so if the registry pod is ever replaced, re-run `make k8s-images`.
## 3. Build and push the images
```bash
make k8s-images K8S_REGISTRY=<TALOS_HOST>:30500
```
This builds the nine images with `docker compose build` — same contexts and Dockerfiles as
compose, no second build definition — then `docker save | crane push --insecure` each one.
`docker push` is not used: the registry speaks plain HTTP, which the Docker daemon refuses
without a root-level `insecure-registries` entry, while crane just takes `--insecure`.
## 4. Deploy
```bash
make k8s-up TALOS_HOST=<TALOS_HOST> K8S_REGISTRY=<TALOS_HOST>:30500
```
That does two things:
1. `make k8s-seed` — creates the ConfigMaps the chart mounts, from the config files that
already live in this repo (`infra/helm/seed-configmaps.sh`): the four
`setup_configuration/data.yaml` files, the Keycloak realm exports, the BPMN + DMN, and
the two bootstrap scripts. Re-run it after editing any of them.
2. `helm upgrade --install` of the chart into namespace `big`.
First bring-up takes a few minutes: the four Django services migrate their databases and
apply their `setup_configuration`, Flowable creates its schema, and the bootstrap Jobs
deploy the BPMN/DMN, seed the zaaktype and register the NRC abonnement.
```bash
kubectl -n big get pods -w
kubectl -n big get jobs # all four must reach COMPLETIONS 1/1
```
The Jobs are the stack's wiring; if one is not complete, the flow is broken somewhere
specific:
| Job | What breaks without it |
|---|---|
| `flowable-init` | no `registratie` process, no diploma DMN |
| `registerrecord-init` | the register has no RegisterRecord objecttype, so writes are refused |
| `seed-zaaktype` | the ACL can't resolve `BIG-REGISTRATIE`, so no zaak is created |
| `nrc-subscribe` | register writes never reach the projection — the public register stays empty |
## 5. Use it
### The portals must be reached over `localhost`
The portals' OIDC flow uses PKCE, which needs `crypto.subtle` — and browsers only expose
that in a **secure context**: HTTPS, or an origin on `localhost`/`127.0.0.1`. A NodePort on
the VM's IP is neither, so `http://<TALOS_HOST>:30140` fails before it can even build the
authorize URL:
```
ERROR TypeError: Cannot read properties of undefined (reading 'digest')
at t.calcHash → t.generateCodeChallenge → t.createUrlCodeFlowAuthorize
```
So deploy with `TALOS_HOST=localhost` — which pins Keycloak's issuer and the portals'
`config.json` authority to `http://localhost:30180` — and forward the browser-facing
services to those same ports:
```bash
make k8s-up TALOS_HOST=localhost K8S_REGISTRY=<TALOS_HOST>:30500
make k8s-portals # stays in the foreground; Ctrl-C stops all five forwards
```
| URL (needs `make k8s-portals`) | What |
|---|---|
| `http://localhost:30140` | self-service portal (DigiD) |
| `http://localhost:30141` | openbaar register (anonymous) |
| `http://localhost:30142` | behandel portal (medewerker) |
| `http://localhost:30143` | beheer portal (medewerker) |
| `http://localhost:30180` | Keycloak (admin/admin) |
The port numbers are deliberately the NodePort numbers: Keycloak's issuer is one fixed
string, so the port the browser uses has to match the one baked into `config.json`.
This is the same mechanism `infra/host-browser.yml` uses for the compose stack (which pins
`localhost:8180`); only the addresses differ.
### The admin UIs work straight off the NodePorts
These are server-rendered and need no secure context, so they are reachable at the VM's
address with no forwarding:
| URL | What |
|---|---|
| `http://<TALOS_HOST>:30000` | OpenZaak admin (admin/admin) |
| `http://<TALOS_HOST>:30001` | Open Notificaties admin (admin/admin) |
| `http://<TALOS_HOST>:30020` / `:30021` | Objecttypen / Objecten admin |
| `http://<TALOS_HOST>:30080` | BFF (`/health`) |
| `http://<TALOS_HOST>:30090` | Flowable REST (rest-admin/test) |
### Credentials
Log in with the test users from `docs/synthetic-data.md` (all password `test123`, e.g.
`jan-burger` for self-service, `merel-behandelaar` for behandel). The `medewerker` realm
enforces MFA (ADR-0031) — print a current code with
`python3 infra/keycloak/check_realms.py otp`. Walk the flow in `docs/demo-script.md`.
`TALOS_HOST` is not cosmetic: it pins Keycloak's issuer (`KC_HOSTNAME`) and the portals'
OIDC authority to the same string, which is what makes a browser token pass the BFF's
validation (ADR-0010). Change it and you must re-run `make k8s-up` — the chart rolls the
portals for you, because their `config.json` is a subPath mount and would otherwise keep
serving the old authority.
### Smoke-test the whole chain without a browser
With the forwards running:
```bash
TOK=$(curl -s -X POST http://localhost:30180/realms/digid/protocol/openid-connect/token \
-d grant_type=password -d client_id=big-portal \
-d username=jan-burger -d password=test123 -d scope=openid | jq -r .access_token)
# through the portal's Caddy, so this also proves the BFF reverse proxy
curl -s -X POST http://localhost:30140/self-service/registrations \
-H "Authorization: Bearer $TOK" -H 'Content-Length: 0'
# → {"registrationId":"…","status":"Ingediend"}
curl -s http://localhost:30141/openbaar/register
# → [{"id":"…","status":"INGEDIEND","reference":"<the registrationId>"}]
```
The second call proves the whole Common Ground path: portal → BFF → domain → Flowable →
ACL → OpenZaak + Objecten → NRC → event-subscriber → projection → openbaar register.
## 6. Keeping the databases (recommended if you iterate on the chart)
By default every database is an `emptyDir`: no CSI driver needed, and the data lives as
long as the pod. Note what that means in practice — **any** change to a database pod's
template (an image policy, an env value, a probe) recreates the pod and wipes it. The stack
then needs its bootstrap re-run:
```bash
make k8s-reseed TALOS_HOST=... K8S_REGISTRY=...
```
which re-runs the four Jobs *and* restarts `event-subscriber` + `projection-api`, because
those two create the projection schema on start and otherwise keep writing to a
schema-less database (`relation "processed_notifications" does not exist`). For persistence, install Rancher's local-path-provisioner — on Talos it
must write under `/var` and its namespace needs the privileged Pod Security label:
```yaml
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- github.com/rancher/local-path-provisioner/deploy?ref=v0.0.31
patches:
- patch: |-
kind: ConfigMap
apiVersion: v1
metadata:
name: local-path-config
namespace: local-path-storage
data:
config.json: |-
{ "nodePathMap":[ { "node":"DEFAULT_PATH_FOR_NON_LISTED_NODES", "paths":["/var/local-path-provisioner"] } ] }
- patch: |-
apiVersion: v1
kind: Namespace
metadata:
name: local-path-storage
labels:
pod-security.kubernetes.io/enforce: privileged
```
```bash
kubectl apply -k .
make k8s-up TALOS_HOST=... K8S_REGISTRY=... K8S_SET='--set persistence.storageClass=local-path'
```
The PVCs carry `helm.sh/resource-policy: keep`, so `make k8s-down` leaves the data behind;
`make k8s-purge` drops the namespace and with it the volumes.
## 7. Day-to-day
```bash
make k8s-lint # render + schema-check the chart, no cluster needed
make k8s-drift # fail if compose and the chart describe different stacks
make k8s-portals # forward the portals + Keycloak to localhost (browser access)
make k8s-images K8S_REGISTRY=... # after changing a service or a portal
make k8s-up TALOS_HOST=... K8S_REGISTRY=...
make k8s-seed # after editing a data.yaml, a realm export, or the BPMN
make k8s-reseed TALOS_HOST=... K8S_REGISTRY=... # re-run the bootstrap Jobs + reset the projection schema
make k8s-down # uninstall, keep the database PVCs
make k8s-purge # uninstall and drop the namespace
```
This repo's images are pulled with `imagePullPolicy: Always` (the `dev` tag is mutable), so
`kubectl -n big rollout restart deploy/<name>` after `make k8s-images` picks up a rebuild.
Upstream images stay `IfNotPresent`: their tags are pinned, and keeping them out of the pod
template avoids needless churn — a changed template makes a Job unpatchable.
`k8s-reseed` is also the path for *changing* a Job in the chart: a Job's pod template is
immutable, so `helm upgrade` is rejected with `cannot patch "…" with kind Job`.
## 8. When it doesn't work
| Symptom | Cause |
|---|---|
| `Boot failed: not a bootable disk` | virt-manager ejected the install ISO on first shutdown — see §1 |
| The VM comes back in maintenance mode after a restart | the ISO is still attached and boots first; eject it and set `--boot hd` (§1) |
| `apply-config` rejects the patch with *"incompatible with v1alpha1"* | Talos ≥1.14 owns that field in its own config document — patch the document, not `machine.*` (§1) |
| CoreDNS `Pending` forever | the control-plane taint is still on the only node (§1) |
| `ImagePullBackOff``pull QPS exceeded` | transient: the kubelet rate-limits pulls when ~30 pods start at once. It recovers on retry |
| `ImagePullBackOff` on a `register-referentie/*` image | the registry mirror patch is missing: `talosctl get registriesconfig` |
| Pod stuck in `ContainerCreating`, event names a ConfigMap | `make k8s-seed` |
| `seed-zaaktype` retrying | publishing a zaaktype validates the resultaattype against `selectielijst.openzaak.nl`, so this one Job needs outbound internet from the VM (ADR-0006) |
| `TypeError: Cannot read properties of undefined (reading 'digest')` on a portal | not a secure context: `crypto.subtle` is absent on `http://<ip>`. Use `localhost` + `make k8s-portals` (§5) |
| Login redirects but the portal stays logged out, or the BFF answers 401 | `TALOS_HOST` doesn't match the address in the browser's URL bar — issuer mismatch. Re-run `make k8s-up` with the right value |
| A portal returns 502 on `/self-service/…` | the BFF is unreachable from the portal pod: check `kubectl -n big get svc bff` and the BFF's own readiness |
| Public register empty after a submit | usually a wiped `emptyDir` database (§6): `make k8s-reseed`. Confirm with `kubectl -n big logs deploy/event-subscriber \| grep 42P01` |
| `helm upgrade` fails with `cannot patch … with kind Job` | see §7 — use `make k8s-reseed` |
| Pods `Evicted` / `OOMKilled` | the VM is too small (§0) |
| A Job shows `BackoffLimitExceeded` | read it: `kubectl -n big logs job/<name>` |
## 9. Deploying on merge to main
`.gitea/workflows/deploy.yaml` runs the §3–§4 steps against the **lab server's** Talos VM
every time a PR is squash-merged to `main` (and on demand via *Run workflow*). PR CI is the
merge gate, so the workflow deploys without re-running the checks.
The cluster's API and registry are not exposed publicly, so the job forwards them over the
same SSH hop the Gitea-runner pipeline uses:
```
ssh -p 6667 user@labs.respellion.tech -L 6443 -L 30500 -L 30141 → <TALOS_VM_IP>
```
Consequences worth knowing:
- Images are **pushed** to `localhost:30500` (the tunnel) and **pulled** by the node from
`<TALOS_VM_IP>:30500` (its own NodePort, the address in the Talos registry-mirror patch).
Same registry, two names — hence the two `K8S_REGISTRY` values in the workflow.
- It calls `make k8s-reseed`, not `make k8s-up`: the bootstrap Jobs are idempotent, and
deleting them first is what stops a changed Job template from wedging `helm upgrade` (§7).
- `dev` is a mutable tag, so a `rollout restart` of the nine repo deployments is what
actually puts the new images in the pods.
- Deploys **queue** (`cancel-in-progress: false`): a helm upgrade killed half-way leaves the
release in `pending-upgrade`, which has to be unwedged by hand.
Settings, all on the repository in Gitea:
| Kind | Name | What |
|---|---|---|
| Secret | `TALOS_SSH_KEY` | private key for `user@labs.respellion.tech` (the Fedora host) |
| Secret | `TALOS_KUBECONFIG` | base64 of the kubeconfig, **`server: https://127.0.0.1:6443`** — Talos puts `127.0.0.1` in the apiserver cert SANs, so TLS still verifies through the tunnel |
| Variable | `TALOS_VM_IP` | the VM's libvirt address (default `192.168.122.173`) |
| Variable | `TALOS_HOST` | the browser-facing host baked into Keycloak's issuer (default `localhost`, see §5) |
The last step smokes `GET /openbaar/register` through the openbaar portal, which exercises
portal → Caddy → BFF → projection. An empty register passes; a 502 does not.
Not covered: the portals still need `make k8s-portals` (or an SSH forward) to be usable in a
browser, because PKCE needs a secure context (§5). Giving the server a hostname + TLS is the
upgrade path.
## What is not ported
- **Observability** (Tempo, Prometheus, Grafana) is defined but disabled — those are built
images too, so switching them on means pushing them as well:
`K8S_SET='--set workloads.tempo.enabled=true --set workloads.prometheus.enabled=true --set workloads.grafana.enabled=true'`.
The .NET services still export OTLP; the exporter fails harmlessly when Tempo is absent.
- **The verify/e2e lanes.** `make verify*` and the Playwright e2e drive compose, not the
chart. The Kubernetes path is verified with §5's smoke test. CI's `k8s` job runs the two
clusterless checks (`k8s-lint`, `k8s-drift`) on every PR — a values typo or a compose
image bump that skipped the chart fails there, but nothing deploys the chart in CI.
- **Ingress, TLS, and resource requests.** See the ponytail ceiling in ADR-0033.
+8
View File
@@ -19,6 +19,11 @@ All test users share the password **`test123`**.
| `eidas` | eIDAS (EU) | `pierre-dupont` | `eidas_id` = `FR/NL/AB-1234-5678` |
| `medewerker` | Internal staff | `merel-behandelaar` | role `behandelaar` |
| `medewerker` | Internal staff | `tom-teamlead` | roles `behandelaar`, `teamlead` |
| `medewerker` | Internal staff | `bram-beheerder` | role `beheerder` |
`medewerker` users additionally need a **second factor**: that realm enforces MFA (S-15c,
ADR-0031). All three share the fixture TOTP secret `BIGMEDEWERKEROTPSEED`; print a current
code with `python3 infra/keycloak/check_realms.py otp`.
The identifying claims are injected via OIDC protocol mappers on `big-portal`
(user-attribute → token claim); `medewerker` roles appear in `realm_access.roles`.
@@ -32,5 +37,8 @@ curl -s -X POST \
-d username=jan-burger -d password=test123 -d scope=openid | jq -r .access_token
```
For a `medewerker` user, add `-d totp=$(python3 infra/keycloak/check_realms.py otp)`
without it the grant is refused with `invalid_grant`.
Decode the JWT payload to see the `bsn` claim. `make keycloak-smoke` checks every realm
automatically.
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env python3
"""Fail when a page under docs/ is missing from mkdocs.yml's nav.
docs/ is the source of truth (CLAUDE.md §12), but only the pages listed in the nav
are published and mkdocs' own `omitted_files: warn` keeps a build green while
silently dropping them, which is how every ADR after 0010 and every runbook but
ci.md fell off the site.
ponytail: a substring test, not a YAML parse a page's path either appears in
mkdocs.yml or it doesn't, and that needs no dependency.
"""
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
nav = (ROOT / "mkdocs.yml").read_text()
missing = sorted(
str(page.relative_to(ROOT / "docs"))
for page in (ROOT / "docs").rglob("*.md")
if str(page.relative_to(ROOT / "docs")) not in nav
)
if missing:
print(f"{len(missing)} page(s) under docs/ are not in mkdocs.yml's nav:")
print("\n".join(f" {m}" for m in missing))
sys.exit(1)
print("docs nav complete: every page under docs/ is published")
+134 -3
View File
@@ -56,6 +56,10 @@ services:
oz-init:
image: docker.io/openzaak/open-zaak:${OPENZAAK_TAG:-1.28.2}
environment: &oz-env
# 1 uWSGI worker, not the image default of 4×4 (#147) — idle workers pressure the runner; the
# -init/-celery containers share this anchor and ignore it (they don't run uwsgi).
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: openzaak.conf.docker
SECRET_KEY: ${OZ_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: oz-db
@@ -138,6 +142,9 @@ services:
# bind-mounted here (this twin is the local/no-make path). See ADR-0007.
image: docker.io/openzaak/open-notificaties:${OPENNOTIFICATIES_TAG:-1.16.1}
environment: &nrc-env
# 1 uWSGI worker, not the image default of 4×4 (#147) — see the oz-env note above.
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: nrc.conf.docker
SECRET_KEY: ${NRC_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: nrc-db
@@ -331,6 +338,15 @@ services:
Acl__Defaults__Vertrouwelijkheidaanduiding: openbaar
Acl__Defaults__ZaaktypeIdentificatie: BIG-REGISTRATIE
Acl__Defaults__InformatieobjecttypeOmschrijving: Diploma
# Objecten holds the register, OpenZaak holds the process (S-19a, ADR-0028). Both APIs take a
# static token, not a ZGW JWT. The objecttype URL is assigned at seed time, so the ACL resolves
# it by name — lazily, on the first approval, so no depends_on is needed here.
# Dotted host on purpose — see the `objecten.local` alias below (ADR-0029).
Acl__Objecten__BaseUrl: http://objecten.local:8000/
Acl__Objecten__Token: ${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}
Acl__Objecten__ObjecttypenBaseUrl: http://objecttypen:8000/
Acl__Objecten__ObjecttypenToken: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
Acl__Objecten__ObjecttypeName: RegisterRecord
ports:
- "8100:8080"
volumes:
@@ -494,7 +510,7 @@ services:
networks: [cg]
# ── Portals (S-08/S-09/S-12) ──────────────────────────────────────────────
# nginx serves each Angular app and reverse-proxies its endpoint group to the BFF (same-origin).
# Caddy serves each Angular app and reverse-proxies its endpoint group to the BFF (same-origin).
# The images bake config.json with the compose authority (keycloak:8080), which a HOST browser
# can't resolve — so here we bind-mount a config.json pointing at the host-published localhost:8180
# (matching KC_HOSTNAME). openbaar is anonymous and needs no config.
@@ -506,7 +522,7 @@ services:
ports:
- "8140:80"
volumes:
- ./local-config/self-service.config.json:/usr/share/nginx/html/config.json:ro,z
- ./local-config/self-service.config.json:/usr/share/caddy/config.json:ro,z
healthcheck:
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
@@ -546,7 +562,7 @@ services:
ports:
- "8142:80"
volumes:
- ./local-config/behandel.config.json:/usr/share/nginx/html/config.json:ro,z
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
healthcheck:
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
@@ -583,6 +599,9 @@ services:
objecttypen-init:
image: docker.io/maykinmedia/objecttypes-api:${OBJECTTYPES_TAG:-3.4.2}
environment: &objecttypen-env-local
# 1 uWSGI worker, not the image default of 4×4 (#144) — idle workers starve the CI runner.
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objecttypes.conf.docker
SECRET_KEY: ${OBJECTTYPES_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: objecttypen-db
@@ -621,12 +640,124 @@ services:
condition: service_completed_successfully
networks: [cg]
# ── RegisterRecord objecttype (S-18c) — API-seeded one-shot (local variant) ─
registerrecord-init:
image: docker.io/library/python:3-slim
environment:
OBJECTTYPEN: http://objecttypen:8000
OBJECTTYPEN_TOKEN: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
SCHEMA: /config/registerrecord.schema.json
command: python /config/register.py
volumes:
- ./objecttypen-registerrecord:/config:ro,z
depends_on:
objecttypen:
condition: service_healthy
networks: [cg]
# ── Objecten API (S-18b) — bind-mounted config (local variant) ─────────────
objecten-db:
image: docker.io/postgis/postgis:17-3.5
environment:
POSTGRES_USER: objects
POSTGRES_PASSWORD: objects
POSTGRES_DB: objects
volumes:
- objecten-db:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U objects"]
interval: 5s
timeout: 3s
retries: 10
networks: [cg]
objecten-redis:
image: docker.io/library/redis:7
networks: [cg]
objecten-init:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: &objecten-env-local
# 1 uWSGI worker, not the image default of 4×4 (#144) — idle workers starve the CI runner.
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objects.conf.docker
SECRET_KEY: ${OBJECTS_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: objecten-db
DB_NAME: objects
DB_USER: objects
DB_PASSWORD: objects
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: objecten-redis:6379/0
CACHE_AXES: objecten-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
CELERY_BROKER_URL: redis://objecten-redis:6379/1
CELERY_RESULT_BACKEND: redis://objecten-redis:6379/1
# Publish register-record events to NRC on the `objecten` kanaal (S-19b-1, ADR-0029). The NRC
# service + notifications_config are provisioned by setup_configuration
# (infra/objecten/setup_configuration/data.yaml), and objecten-celery below actually sends
# them — notifications_api_common only queues the task. See ADR-0028 for why S-19a left this
# off until all four pieces existed.
NOTIFICATIONS_DISABLED: "false"
RUN_SETUP_CONFIG: "true"
command: /setup_configuration.sh
volumes:
- ./objecten/setup_configuration:/app/setup_configuration:ro,z
depends_on:
objecten-db:
condition: service_healthy
objecten-redis:
condition: service_started
objecttypen:
condition: service_healthy
networks: [cg]
objecten:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env-local
healthcheck:
test: ["CMD", "python", "-c", "import requests,sys; sys.exit(0 if requests.head('http://localhost:8000/admin/').status_code in (200,302) else 1)"]
interval: 10s
timeout: 5s
retries: 10
start_period: 30s
ports:
- "8021:8000"
depends_on:
objecten-init:
condition: service_completed_successfully
networks:
cg:
# Objecten reflects the *request* Host into the `url` it returns, and
# notifications_api_common publishes that url as the notification's hoofdObject /
# resourceUrl — which NRC types as a URLField, and Django's URLValidator rejects a
# single-label host ("Voer een geldige URL in."). So every caller whose writes must be
# notified addresses Objecten by this dotted alias instead of `objecten` (ADR-0029).
# Reads are unaffected and still use the plain service name.
aliases:
- objecten.local
# The celery worker that actually delivers Objecten's notifications to NRC (S-19b-1, ADR-0029).
# notifications_api_common only schedules the send on transaction commit; without a worker the
# task sits in redis forever and every register write is silently undelivered. Mirrors oz-celery.
# No beat: Objecten is a publisher, not a subscriber — nrc-beat drains the delivery queue.
objecten-celery:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env-local
command: /celery_worker.sh
depends_on:
objecten-init:
condition: service_completed_successfully
networks: [cg]
volumes:
oz-db:
nrc-db:
flowable-db:
projection-db:
objecttypen-db:
objecten-db:
# Carries the seed-generated acl.env (server-assigned zaaktype URLs) from local-seed to the ACL.
seed-env:
+160 -8
View File
@@ -51,6 +51,12 @@ services:
oz-init:
image: docker.io/openzaak/open-zaak:${OPENZAAK_TAG:-1.28.2}
environment: &oz-env
# 1 uWSGI worker, not the image default of 4×4 (#147, same lever as #145): OpenZaak serves
# single-request smoke checks here and is not load-tested, so 4 idle Django workers just pin
# ~800 MB and pressure the shared runner. The -init (setup_configuration) and -celery containers
# share this anchor and ignore it — they don't run uwsgi.
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: openzaak.conf.docker
SECRET_KEY: ${OZ_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: oz-db
@@ -135,6 +141,9 @@ services:
# needs no baked config.
image: docker.io/openzaak/open-notificaties:${OPENNOTIFICATIES_TAG:-1.16.1}
environment: &nrc-env
# 1 uWSGI worker, not the image default of 4×4 (#147) — see the oz-env note above.
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: nrc.conf.docker
SECRET_KEY: ${NRC_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: nrc-db
@@ -314,6 +323,15 @@ services:
# so verify-domain still points the ACL at OpenZaak's container IP.
Acl__Defaults__ZaaktypeIdentificatie: BIG-REGISTRATIE
Acl__Defaults__InformatieobjecttypeOmschrijving: Diploma
# Objecten holds the register, OpenZaak holds the process (S-19a, ADR-0028). Both APIs take a
# static token, not a ZGW JWT. The objecttype URL is assigned at seed time, so the ACL resolves
# it by name — lazily, on the first approval, so no depends_on is needed here.
# Dotted host on purpose — see the `objecten.local` alias below (ADR-0029).
Acl__Objecten__BaseUrl: http://objecten.local:8000/
Acl__Objecten__Token: ${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}
Acl__Objecten__ObjecttypenBaseUrl: http://objecttypen:8000/
Acl__Objecten__ObjecttypenToken: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
Acl__Objecten__ObjecttypeName: RegisterRecord
ports:
- "8100:8080"
healthcheck:
@@ -478,7 +496,7 @@ services:
networks: [cg]
# ── Self-Service portal (S-08d) ────────────────────────────────────────────
# nginx serves the Angular app and reverse-proxies /self-service + /openbaar to the BFF
# Caddy serves the Angular app and reverse-proxies /self-service + /openbaar to the BFF
# (same-origin, no CORS). The Playwright e2e drives it inside this network so the DigiD
# token issuer (keycloak:8080) matches the BFF's authority (ADR-0010).
self-service:
@@ -489,7 +507,7 @@ services:
ports:
- "8140:80"
healthcheck:
# 127.0.0.1, not localhost: nginx listens on IPv4 only, but localhost resolves to ::1 first.
# 127.0.0.1, not localhost: keeps the check on the interface Caddy is published on.
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
timeout: 3s
@@ -502,7 +520,7 @@ services:
condition: service_started
networks: [cg]
# The openbaar (public) register portal: nginx serves the Angular app and reverse-proxies
# The openbaar (public) register portal: Caddy serves the Angular app and reverse-proxies
# /openbaar to the BFF. Anonymous — no DigiD, no Keycloak dependency (S-09).
openbaar:
build:
@@ -512,7 +530,7 @@ services:
ports:
- "8141:80"
healthcheck:
# 127.0.0.1, not localhost: nginx listens on IPv4 only, but localhost resolves to ::1 first.
# 127.0.0.1, not localhost: keeps the check on the interface Caddy is published on.
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
timeout: 3s
@@ -523,7 +541,7 @@ services:
condition: service_healthy
networks: [cg]
# The behandel portal: nginx serves the Angular app and reverse-proxies /behandel to the BFF.
# The behandel portal: Caddy serves the Angular app and reverse-proxies /behandel to the BFF.
# Behandelaars log in against the Keycloak medewerker realm (ADR-0013; S-12).
behandel:
build:
@@ -533,7 +551,7 @@ services:
ports:
- "8142:80"
healthcheck:
# 127.0.0.1, not localhost: nginx listens on IPv4 only, but localhost resolves to ::1 first.
# 127.0.0.1, not localhost: keeps the check on the interface Caddy is published on.
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
timeout: 3s
@@ -546,7 +564,7 @@ services:
condition: service_started
networks: [cg]
# The beheer portal: nginx serves the Angular app and reverse-proxies /beheer to the BFF.
# The beheer portal: Caddy serves the Angular app and reverse-proxies /beheer to the BFF.
# Beheerders log in against the Keycloak medewerker realm (same realm as behandel, S-15a).
beheer:
build:
@@ -556,7 +574,7 @@ services:
ports:
- "8143:80"
healthcheck:
# 127.0.0.1, not localhost: nginx listens on IPv4 only, but localhost resolves to ::1 first.
# 127.0.0.1, not localhost: keeps the check on the interface Caddy is published on.
test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1/ || exit 1"]
interval: 5s
timeout: 3s
@@ -596,6 +614,11 @@ services:
objecttypen-init:
image: docker.io/maykinmedia/objecttypes-api:${OBJECTTYPES_TAG:-3.4.2}
environment: &objecttypen-env
# 1 uWSGI worker, not the image default of 4×4: this API only serves single-request smoke
# checks and sits idle during the e2e step — 4 idle Django workers each pin ~200 MB and starve
# the shared CI runner (#144). Init ignores this (it runs setup_configuration, not uwsgi).
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objecttypes.conf.docker
SECRET_KEY: ${OBJECTTYPES_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: objecttypen-db
@@ -635,6 +658,128 @@ services:
condition: service_completed_successfully
networks: [cg]
# ── RegisterRecord objecttype (S-18c) — API-seeded one-shot ────────────────
# The Objecttypen setup_configuration (3.4.2) can only provision tokens — no declarative objecttype
# step — so this one-shot creates the RegisterRecord objecttype + a published version over the API
# once Objecttypen is healthy (idempotent; ADR-0020 self-seed, ADR-0027 schema). The schema + script
# are streamed into the external config volume by infra/seed-config.sh, like the *-init volumes.
registerrecord-init:
image: docker.io/library/python:3-slim
environment:
OBJECTTYPEN: http://objecttypen:8000
OBJECTTYPEN_TOKEN: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
SCHEMA: /config/registerrecord.schema.json
command: python /config/register.py
volumes:
- registerrecord-config:/config:ro
depends_on:
objecttypen:
condition: service_healthy
networks: [cg]
# ── Objecten API (S-18b) — upstream Maykin image, verbatim ─────────────────
# The authoritative object store. Same shape as Objecttypen (own DB + redis, an `-init` that runs
# setup_configuration from the external config volume, a health-checked web). Two differences: the
# DB is PostGIS (objects carry geometry), and setup_configuration registers the Objecttypen API
# (S-18a) as a trusted service so an object can reference its objecttype.
objecten-db:
image: docker.io/postgis/postgis:17-3.5
environment:
POSTGRES_USER: objects
POSTGRES_PASSWORD: objects
POSTGRES_DB: objects
volumes:
- objecten-db:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U objects"]
interval: 5s
timeout: 3s
retries: 10
networks: [cg]
objecten-redis:
image: docker.io/library/redis:7
networks: [cg]
objecten-init:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: &objecten-env
# 1 uWSGI worker, not the image default of 4×4 — see the objecttypen note above (#144).
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objects.conf.docker
SECRET_KEY: ${OBJECTS_SECRET_KEY:-dev-only-not-for-production}
DB_HOST: objecten-db
DB_NAME: objects
DB_USER: objects
DB_PASSWORD: objects
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: objecten-redis:6379/0
CACHE_AXES: objecten-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
CELERY_BROKER_URL: redis://objecten-redis:6379/1
CELERY_RESULT_BACKEND: redis://objecten-redis:6379/1
# Publish register-record events to NRC on the `objecten` kanaal (S-19b-1, ADR-0029). The NRC
# service + notifications_config are provisioned by setup_configuration
# (infra/objecten/setup_configuration/data.yaml), and objecten-celery below actually sends
# them — notifications_api_common only queues the task. See ADR-0028 for why S-19a left this
# off until all four pieces existed.
NOTIFICATIONS_DISABLED: "false"
RUN_SETUP_CONFIG: "true"
command: /setup_configuration.sh
# data.yaml is streamed into this external volume by infra/seed-config.sh before start.
volumes:
- objecten-config:/app/setup_configuration:ro
depends_on:
objecten-db:
condition: service_healthy
objecten-redis:
condition: service_started
# Objecten's setup_configuration registers the Objecttypen service; that service only needs to
# exist as config, but wait for Objecttypen to be up so the register is meaningful end to end.
objecttypen:
condition: service_healthy
networks: [cg]
objecten:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env
healthcheck:
test: ["CMD", "python", "-c", "import requests,sys; sys.exit(0 if requests.head('http://localhost:8000/admin/').status_code in (200,302) else 1)"]
interval: 10s
timeout: 5s
retries: 10
start_period: 30s
ports:
- "8021:8000"
depends_on:
objecten-init:
condition: service_completed_successfully
networks:
cg:
# Objecten reflects the *request* Host into the `url` it returns, and
# notifications_api_common publishes that url as the notification's hoofdObject /
# resourceUrl — which NRC types as a URLField, and Django's URLValidator rejects a
# single-label host ("Voer een geldige URL in."). So every caller whose writes must be
# notified addresses Objecten by this dotted alias instead of `objecten` (ADR-0029).
# Reads are unaffected and still use the plain service name.
aliases:
- objecten.local
# The celery worker that actually delivers Objecten's notifications to NRC (S-19b-1, ADR-0029).
# notifications_api_common only schedules the send on transaction commit; without a worker the
# task sits in redis forever and every register write is silently undelivered. Mirrors oz-celery.
# No beat: Objecten is a publisher, not a subscriber — nrc-beat drains the delivery queue.
objecten-celery:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env
command: /celery_worker.sh
depends_on:
objecten-init:
condition: service_completed_successfully
networks: [cg]
# ── Observability backplane (S-16a, ADR-0023) ──────────────────────────────
# Grafana-native stack: Tempo ingests OTLP traces (the .NET services export
# straight to it — no collector hop, S-16b), Prometheus scrapes service
@@ -685,6 +830,7 @@ volumes:
flowable-db:
projection-db:
objecttypen-db:
objecten-db:
# Config volumes — created and populated out-of-band by infra/seed-config.sh
# (docker cp), because bind mounts don't reach sibling containers on the CI
# runner. `external` keeps the names deterministic; the seed step manages them.
@@ -703,6 +849,12 @@ volumes:
objecttypen-config:
external: true
name: rr-objecttypen-config
registerrecord-config:
external: true
name: rr-registerrecord-config
objecten-config:
external: true
name: rr-objecten-config
networks:
cg:
+8
View File
@@ -0,0 +1,8 @@
apiVersion: v2
name: big-reference
description: >-
The BIG reference stack (Common Ground) on Kubernetes — a port of
infra/docker-compose.yml, aimed at a single-node Talos cluster.
type: application
version: 0.1.0
appVersion: dev
@@ -0,0 +1,25 @@
{{ .Chart.Name }} {{ .Chart.Version }} deployed to namespace {{ .Release.Namespace }}.
Watch it converge (the upstream Django services migrate on first boot, so the
first bring-up takes a few minutes):
kubectl -n {{ .Release.Namespace }} get pods -w
kubectl -n {{ .Release.Namespace }} get jobs
Every bootstrap Job must reach Completions 1/1:
{{- range $name, $w := .Values.workloads }}
{{- if and (ne $w.enabled false) $w.job }}
- {{ $name }}
{{- end }}
{{- end }}
Open in a browser (add {{ .Values.host }} to /etc/hosts if you use a name):
{{- range $name, $port := .Values.nodePorts }}
{{- $w := index $.Values.workloads $name }}
{{- if ne $w.enabled false }}
{{ printf "%-16s http://%s:%v" $name $.Values.host $port }}
{{- end }}
{{- end }}
Test users are in docs/synthetic-data.md. If a pod is stuck in
ContainerCreating on a missing ConfigMap, run: make k8s-seed
@@ -0,0 +1,142 @@
{{/*
One pod spec for every workload, Deployment and Job alike. The chart is
values-driven on purpose: `.Values.workloads` is a near-literal transcription of
infra/docker-compose.yml, so the two stacks can be diffed by eye instead of by
archaeology. Adding a service is a values edit, not a template edit.
Called as: include "big.podspec" (dict "root" $ "name" $name "w" $w)
*/}}
{{- define "big.podspec" -}}
{{- $root := .root -}}
{{- $name := .name -}}
{{- $w := .w -}}
{{- with $root.Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 2 }}
{{- end }}
{{- with $w.waitFor }}
initContainers:
- name: wait-for-deps
image: {{ $root.Values.images.busybox }}
command:
- sh
- -c
- |
for t in {{ join " " . }}; do
echo "waiting for $t"
until nc -z "${t%:*}" "${t#*:}"; do sleep 2; done
done
{{- end }}
containers:
- name: {{ $name }}
image: {{ include "big.image" (dict "root" $root "name" $name "w" $w) }}
# Only this repo's images get the configured policy: their `dev` tag is mutable.
# Upstream tags are pinned, so IfNotPresent keeps them out of pod-template diffs —
# which matters because a changed template makes a Job unpatchable (immutable).
imagePullPolicy: {{ if $w.own }}{{ $root.Values.images.pullPolicy }}{{ else }}IfNotPresent{{ end }}
{{- if $w.command }}
{{- fail (printf "workload %s: use `args`, not `command` — compose's `command:` replaces CMD, but Kubernetes' `command:` replaces the image ENTRYPOINT (postgres would run as root, keycloak would exec `start-dev`)" $name) }}
{{- end }}
{{- with $w.args }}
args:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- with $w.envFrom }}
envFrom:
{{- range . }}
- configMapRef:
# optional: an env group whose feature is disabled (e.g. otel) simply
# isn't rendered, and the pod must still start.
name: {{ printf "%s-env" . }}
optional: true
{{- end }}
{{- end }}
{{- with $w.env }}
env:
{{- include "big.env" (list $root .) | nindent 6 }}
{{- end }}
{{- with $w.ports }}
ports:
{{- range . }}
- name: {{ .name }}
containerPort: {{ .targetPort | default .port }}
{{- end }}
{{- end }}
{{- with $w.probe }}
readinessProbe:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- with $w.resources }}
resources:
{{- toYaml . | nindent 6 }}
{{- end }}
{{- if or $w.files $w.data }}
volumeMounts:
{{- range $w.files }}
- name: {{ .configMap }}
mountPath: {{ .mountPath }}
{{- with .subPath }}
subPath: {{ . }}
{{- end }}
readOnly: true
{{- end }}
{{- with $w.data }}
- name: data
mountPath: {{ .mountPath }}
{{- end }}
{{- end }}
{{- if or $w.files $w.data }}
volumes:
{{- range $w.files }}
- name: {{ .configMap }}
configMap:
name: {{ .configMap }}
{{- with .defaultMode }}
defaultMode: {{ . }}
{{- end }}
{{- end }}
{{- with $w.data }}
- name: data
{{- if $root.Values.persistence.storageClass }}
persistentVolumeClaim:
claimName: {{ $name }}-data
{{- else }}
# No StorageClass configured: the databases are emptyDir, so the stack needs
# no CSI driver to come up. Data then lives as long as the pod does — see
# docs/runbooks/kubernetes-talos.md for switching on local-path.
emptyDir: {}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
{{/* Image ref: `own: true` workloads are built from this repo, everything else is upstream. */}}
{{- define "big.image" -}}
{{- $root := .root -}}
{{- $w := .w -}}
{{- if $w.own -}}
{{- $ref := printf "%s/%s:%s" $root.Values.images.repositoryPrefix .name $root.Values.images.tag -}}
{{- with $root.Values.images.registry }}{{ printf "%s/%s" . $ref }}{{ else }}{{ $ref }}{{ end }}
{{- else -}}
{{- $w.image -}}
{{- end -}}
{{- end -}}
{{/*
Env list from a map. Every value is run through `tpl`, so values.yaml can name
cluster-internal hosts ({{ .Release.Namespace }}) and the node address
({{ .Values.host }}) without the chart hard-coding either.
*/}}
{{- define "big.env" -}}
{{- $root := index . 0 -}}
{{- range $k, $v := index . 1 }}
- name: {{ $k }}
value: {{ tpl (toString $v) $root | quote }}
{{- end }}
{{- end -}}
{{- define "big.labels" -}}
app.kubernetes.io/name: {{ .name }}
app.kubernetes.io/instance: {{ .root.Release.Name }}
app.kubernetes.io/managed-by: Helm
{{- end -}}
@@ -0,0 +1,44 @@
{{- /*
Shared env blocks — the Kubernetes equivalent of the YAML anchors in
infra/docker-compose.yml (&oz-env, &nrc-env, &objecttypen-env, &objecten-env).
A workload picks them up with `envFrom`, so the web/celery/init variants of an
upstream image stay guaranteed-identical, and `kubectl get cm oz-env -o yaml`
shows what a pod actually got.
The *file* inputs (setup_configuration data.yaml, Keycloak realms, BPMN/DMN, the
seed scripts) are NOT here: they live in the repo and are turned into ConfigMaps
by infra/helm/seed-configmaps.sh, exactly as infra/seed-config.sh streams them
into the compose config volumes. Copying them into the chart would fork them.
*/ -}}
{{- range $group, $env := .Values.envGroups }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ $group }}-env
labels:
{{- include "big.labels" (dict "root" $ "name" (printf "%s-env" $group)) | nindent 4 }}
data:
{{- range $k, $v := $env }}
{{ $k }}: {{ tpl (toString $v) $ | quote }}
{{- end }}
{{- end }}
{{- /*
Portal OIDC config. The images bake config.json with the compose authority
(keycloak:8080), which a browser outside the cluster cannot resolve; these
ConfigMaps mount over it with the node address Keycloak's issuer is pinned to
(KC_HOSTNAME below), so the token the browser gets and the issuer the BFF
discovers are the same string. Same mechanism as infra/host-browser.yml.
*/ -}}
{{- range $realm := list "digid" "medewerker" }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: portal-config-{{ $realm }}
labels:
{{- include "big.labels" (dict "root" $ "name" (printf "portal-config-%s" $realm)) | nindent 4 }}
data:
config.json: |
{ "authority": "{{ printf "http://%s:%v" $.Values.host (index $.Values.nodePorts "keycloak") }}/realms/{{ $realm }}" }
{{- end }}
@@ -0,0 +1,39 @@
{{- range $name, $w := .Values.workloads }}
{{- if and (ne $w.enabled false) (not $w.job) }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $name }}
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 4 }}
spec:
replicas: 1
# Recreate, not RollingUpdate: single node, ReadWriteOnce volumes, and nothing
# here is HA — a second pod would just fight the first for the disk.
strategy:
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: {{ $name }}
app.kubernetes.io/instance: {{ $.Release.Name }}
template:
metadata:
{{- /*
A ConfigMap mounted with subPath never picks up updates, so a portal whose
config.json content changed has to be rolled. Hashing only the values that
render it keeps the churn off the databases — an emptyDir database that is
recreated for no reason loses its data (see the runbook §6).
*/}}
{{- range $w.files }}
{{- if hasPrefix "portal-config-" .configMap }}
annotations:
checksum/portal-config: {{ printf "%s|%v" $.Values.host (index $.Values.nodePorts "keycloak") | sha256sum }}
{{- end }}
{{- end }}
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 8 }}
spec:
{{- include "big.podspec" (dict "root" $ "name" $name "w" $w) | nindent 6 }}
{{- end }}
{{- end }}
@@ -0,0 +1,29 @@
{{- /*
The one-shot bootstrap containers from compose (oz-init, nrc-init, flowable-init,
the *-init setup_configuration runs, the zaaktype seed and the NRC abonnement)
become Jobs. All of them are idempotent, so ordering is not enforced with hooks:
each waits for the ports it needs (waitFor) and Kubernetes retries the rest.
A wiped database is re-seeded by `make k8s-reseed`.
*/ -}}
{{- range $name, $w := .Values.workloads }}
{{- if and (ne $w.enabled false) $w.job }}
---
apiVersion: batch/v1
kind: Job
metadata:
name: {{ $name }}
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 4 }}
app.kubernetes.io/component: init
spec:
backoffLimit: 20
template:
metadata:
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 8 }}
app.kubernetes.io/component: init
spec:
restartPolicy: OnFailure
{{- include "big.podspec" (dict "root" $ "name" $name "w" $w) | nindent 6 }}
{{- end }}
{{- end }}
@@ -0,0 +1,22 @@
{{- if .Values.persistence.storageClass }}
{{- range $name, $w := .Values.workloads }}
{{- if and (ne $w.enabled false) $w.data }}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ $name }}-data
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 4 }}
# Keep the databases when the release is uninstalled; `make k8s-purge` drops them.
annotations:
helm.sh/resource-policy: keep
spec:
accessModes: [ReadWriteOnce]
storageClassName: {{ $.Values.persistence.storageClass }}
resources:
requests:
storage: {{ $w.data.size | default "2Gi" }}
{{- end }}
{{- end }}
{{- end }}
@@ -0,0 +1,35 @@
{{- /*
Service names are the compose service names, verbatim: the portals' Caddy
proxies to http://bff:8080 and the upstream setup_configuration files name
http://openzaak:8000 / http://nrc-web:8000, so in-cluster DNS has to answer to
exactly those names. Do not rename a workload without checking both.
.Values.nodePorts is the single place a port is published outside the cluster;
a workload listed there gets a NodePort on its first (only) port.
*/ -}}
{{- range $name, $w := .Values.workloads }}
{{- if and (ne $w.enabled false) $w.ports }}
{{- $nodePort := index $.Values.nodePorts $name }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ $name }}
labels:
{{- include "big.labels" (dict "root" $ "name" $name) | nindent 4 }}
spec:
type: {{ if $nodePort }}NodePort{{ else }}ClusterIP{{ end }}
selector:
app.kubernetes.io/name: {{ $name }}
app.kubernetes.io/instance: {{ $.Release.Name }}
ports:
{{- range $i, $p := $w.ports }}
- name: {{ $p.name }}
port: {{ $p.port }}
targetPort: {{ $p.targetPort | default $p.port }}
{{- if and $nodePort (eq $i 0) }}
nodePort: {{ $nodePort }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
+608
View File
@@ -0,0 +1,608 @@
# Values for the BIG reference stack on Kubernetes.
#
# `workloads` is a near-literal transcription of infra/docker-compose.yml — same
# service names, same images, same env, same one-shots — so the two stacks can be
# diffed by eye. Read that file's comments for the *why* behind each setting; only
# the deviations forced by Kubernetes are re-explained here.
#
# Every env value is rendered with Helm's `tpl`, so it may use:
# {{ .Release.Namespace }} — for a cluster-internal FQDN
# {{ .Values.host }} — the node address a browser reaches the cluster on
#
# Deviations from compose, all of them consequences of the platform:
# * The compose stack hands the ACL and the seeds OpenZaak's *container IP*,
# because OpenZaak and NRC validate URLs with Django's URLValidator and a
# single-label host ("openzaak") is rejected. In Kubernetes the service FQDN
# (openzaak.<ns>.svc.cluster.local) is already multi-label, so the IP dance and
# the `objecten.local` network alias both disappear.
# * `depends_on: service_healthy` becomes a `waitFor` init container (TCP wait)
# plus readiness probes. Ordering is otherwise not enforced: every bootstrap
# job is idempotent and Kubernetes retries.
# * The published ports are NodePorts (see `nodePorts`), not host ports.
# The address a browser outside the cluster uses to reach the node: your Talos
# VM's IP. It pins Keycloak's issuer and the portals' OIDC authority to one
# string, so browser tokens and the BFF's discovered issuer agree.
host: 192.168.122.100
# Set when pulling from a private registry (e.g. the Gitea Container Registry).
imagePullSecrets: []
images:
# Where the images built from THIS repo live. Empty = the bare
# `register-referentie/<svc>:dev` names, which only works if the node already
# has them. On Talos it never does — point this at a registry the node can
# reach (see docs/runbooks/kubernetes-talos.md).
registry: ""
repositoryPrefix: register-referentie
tag: dev
# Applies to this repo's images only (see _helpers.tpl). Always, because `dev`
# is a mutable tag: with IfNotPresent the node keeps the first image it pulled
# and `make k8s-images` would appear to do nothing. The registry is in-cluster,
# so a re-pull is local and cheap — but the pods do depend on it being up.
pullPolicy: Always
busybox: docker.io/library/busybox:stable
persistence:
# Empty = every database is an emptyDir, so the stack comes up on a bare
# cluster with no CSI driver. Set to a StorageClass (e.g. `local-path`) to keep
# the data across pod restarts.
storageClass: ""
# The only place a port is published outside the cluster. A workload listed here
# gets a NodePort on its single port; everything else stays ClusterIP.
nodePorts:
openzaak: 30000
nrc-web: 30001
objecttypen: 30020
objecten: 30021
bff: 30080
flowable-rest: 30090
self-service: 30140
openbaar: 30141
behandel: 30142
beheer: 30143
keycloak: 30180
grafana: 30300
# ── Shared env blocks (the compose YAML anchors) ────────────────────────────────
envGroups:
oz:
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: openzaak.conf.docker
SECRET_KEY: dev-only-not-for-production
DB_HOST: oz-db
DB_NAME: openzaak
DB_USER: openzaak
DB_PASSWORD: openzaak
IS_HTTPS: "no"
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: oz-redis:6379/0
CACHE_AXES: oz-redis:6379/0
CELERY_BROKER_URL: redis://oz-redis:6379/1
CELERY_RESULT_BACKEND: redis://oz-redis:6379/1
DISABLE_2FA: "true"
NOTIFICATIONS_DISABLED: "false"
OPENZAAK_SUPERUSER_USERNAME: admin
DJANGO_SUPERUSER_PASSWORD: admin
OPENZAAK_SUPERUSER_EMAIL: admin@localhost
RUN_SETUP_CONFIG: "true"
nrc:
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: nrc.conf.docker
SECRET_KEY: dev-only-not-for-production
DB_HOST: nrc-db
DB_NAME: opennotificaties
DB_USER: opennotificaties
DB_PASSWORD: opennotificaties
IS_HTTPS: "no"
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: nrc-redis:6379/0
CACHE_AXES: nrc-redis:6379/0
CELERY_BROKER_URL: redis://nrc-redis:6379/1
CELERY_RESULT_BACKEND: redis://nrc-redis:6379/1
DISABLE_2FA: "true"
OPENNOTIFICATIES_SUPERUSER_USERNAME: admin
DJANGO_SUPERUSER_PASSWORD: admin
OPENNOTIFICATIES_SUPERUSER_EMAIL: admin@localhost
RUN_SETUP_CONFIG: "true"
NOTIFICATION_SEC_INTERVAL: "5"
objecttypen:
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objecttypes.conf.docker
SECRET_KEY: dev-only-not-for-production
DB_HOST: objecttypen-db
DB_NAME: objecttypes
DB_USER: objecttypes
DB_PASSWORD: objecttypes
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: objecttypen-redis:6379/0
CACHE_AXES: objecttypen-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
RUN_SETUP_CONFIG: "true"
objecten:
UWSGI_PROCESSES: "1"
UWSGI_THREADS: "2"
DJANGO_SETTINGS_MODULE: objects.conf.docker
SECRET_KEY: dev-only-not-for-production
DB_HOST: objecten-db
DB_NAME: objects
DB_USER: objects
DB_PASSWORD: objects
ALLOWED_HOSTS: "*"
CACHE_DEFAULT: objecten-redis:6379/0
CACHE_AXES: objecten-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
CELERY_BROKER_URL: redis://objecten-redis:6379/1
CELERY_RESULT_BACKEND: redis://objecten-redis:6379/1
NOTIFICATIONS_DISABLED: "false"
RUN_SETUP_CONFIG: "true"
# Traces for the .NET services. Always set, like compose: the exporter fails
# harmlessly when Tempo is absent (services/*/Program.cs).
otel:
OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317
OTEL_EXPORTER_OTLP_PROTOCOL: grpc
# ── Workloads ──────────────────────────────────────────────────────────────────
# Per entry: image | own (built here) · args · envFrom (env groups) · env
# ports · probe (a literal readinessProbe) · files (ConfigMap mounts) · data
# (a database volume) · waitFor (host:port to wait for) · job · enabled
#
# `args` (never `command`) is the compose `command:` equivalent: compose replaces
# the image's CMD, and so does Kubernetes' `args` — Kubernetes' `command` would
# replace the ENTRYPOINT instead. The chart fails to render if you use `command`.
workloads:
# ── OpenZaak (S-01) ─────────────────────────────────────────────────────────
oz-db:
image: docker.io/postgis/postgis:17-3.5
args: [postgres, -c, max_connections=300]
env:
POSTGRES_USER: openzaak
POSTGRES_PASSWORD: openzaak
POSTGRES_DB: openzaak
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 4Gi }
probe:
exec:
command: [sh, -c, "pg_isready -U openzaak -d openzaak && psql -U openzaak -d openzaak -c 'SELECT PostGIS_Version();' -q"]
periodSeconds: 5
oz-redis:
image: docker.io/library/redis:7
ports: [{ name: redis, port: 6379 }]
probe: { tcpSocket: { port: 6379 } }
openzaak:
image: docker.io/openzaak/open-zaak:1.28.2
# setup_configuration first, then the server — in ONE container, on purpose.
# Both /setup_configuration.sh and /start.sh run `manage.py migrate`, so a
# separate init Job (as compose has, ordered by depends_on) races this pod for
# the same database and Django fails with "relation already exists".
args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]
envFrom: [oz]
ports: [{ name: http, port: 8000 }]
# /admin/ answers 302 when Django is up — a redirect counts as ready.
probe:
httpGet: { path: /admin/, port: 8000 }
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 30
files: [{ configMap: rr-oz-config, mountPath: /app/setup_configuration }]
waitFor: [oz-db:5432, oz-redis:6379]
oz-celery:
image: docker.io/openzaak/open-zaak:1.28.2
args: [/celery_worker.sh]
envFrom: [oz]
waitFor: [oz-db:5432, oz-redis:6379]
# ── Open Notificaties / NRC (S-01-c) ────────────────────────────────────────
nrc-db:
image: docker.io/postgis/postgis:17-3.5
args: [postgres, -c, max_connections=300]
env:
POSTGRES_USER: opennotificaties
POSTGRES_PASSWORD: opennotificaties
POSTGRES_DB: opennotificaties
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 2Gi }
probe:
exec: { command: [pg_isready, -U, opennotificaties, -d, opennotificaties] }
periodSeconds: 5
nrc-redis:
image: docker.io/library/redis:7
ports: [{ name: redis, port: 6379 }]
probe: { tcpSocket: { port: 6379 } }
nrc-web:
image: docker.io/openzaak/open-notificaties:1.16.1
# setup_configuration first, then the server — in ONE container, on purpose.
# Both /setup_configuration.sh and /start.sh run `manage.py migrate`, so a
# separate init Job (as compose has, ordered by depends_on) races this pod for
# the same database and Django fails with "relation already exists".
args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]
envFrom: [nrc]
ports: [{ name: http, port: 8000 }]
probe:
httpGet: { path: /admin/, port: 8000 }
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 30
files: [{ configMap: rr-nrc-config, mountPath: /app/setup_configuration }]
waitFor: [nrc-db:5432, nrc-redis:6379, openzaak:8000]
nrc-celery:
image: docker.io/openzaak/open-notificaties:1.16.1
args: [/celery_worker.sh]
envFrom: [nrc]
waitFor: [nrc-db:5432, nrc-redis:6379]
# Without beat, notifications are accepted but never delivered (ADR-0007).
nrc-beat:
image: docker.io/openzaak/open-notificaties:1.16.1
args: [/celery_beat.sh]
envFrom: [nrc]
waitFor: [nrc-db:5432, nrc-redis:6379]
# ── Keycloak (S-02) ─────────────────────────────────────────────────────────
keycloak:
image: quay.io/keycloak/keycloak:26.1
args: [start-dev, --import-realm]
env:
KC_BOOTSTRAP_ADMIN_USERNAME: admin
KC_BOOTSTRAP_ADMIN_PASSWORD: admin
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_HEALTH_ENABLED: "true"
KC_HTTP_ENABLED: "true"
# Pin the issuer to the address the browser uses, and let backchannel calls
# keep using keycloak:8080 — the BFF discovers metadata in-cluster and gets
# this issuer back, which is what browser tokens carry (infra/host-browser.yml).
KC_HOSTNAME: "http://{{ .Values.host }}:{{ index .Values.nodePorts \"keycloak\" }}"
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
ports: [{ name: http, port: 8080 }]
# TCP, not /health/ready on the management port: nothing here gates on realm
# import, and a wrong health path would leave the Service with no endpoints.
probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 15 }
files: [{ configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }]
# ── Flowable (S-03) ─────────────────────────────────────────────────────────
flowable-db:
image: docker.io/library/postgres:16
env:
POSTGRES_USER: flowable
POSTGRES_PASSWORD: flowable
POSTGRES_DB: flowable
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 2Gi }
probe:
exec: { command: [pg_isready, -U, flowable, -d, flowable] }
periodSeconds: 5
flowable-rest:
image: docker.io/flowable/flowable-rest:latest
env:
SPRING_DATASOURCE_DRIVER-CLASS-NAME: org.postgresql.Driver
SPRING_DATASOURCE_URL: jdbc:postgresql://flowable-db:5432/flowable
SPRING_DATASOURCE_USERNAME: flowable
SPRING_DATASOURCE_PASSWORD: flowable
ports: [{ name: http, port: 8080 }]
# Every REST path needs basic auth, so an httpGet probe would read 401 as
# not-ready. TCP is the honest signal here.
probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 20 }
waitFor: [flowable-db:5432]
# Deploys the BPMN to the process engine and the DMN to the DMN engine as two
# separate deployments — flowable-rest does not cascade one into the other
# (S-13, ADR-0016). Idempotent.
flowable-init:
job: true
image: docker.io/curlimages/curl:latest
args:
- sh
- -c
- |
svc=http://flowable-rest:8080/flowable-rest/service/repository/deployments
dmn=http://flowable-rest:8080/flowable-rest/dmn-api/dmn-repository/deployments
until curl -sf -u rest-admin:test "$svc" >/dev/null 2>&1; do echo "waiting for flowable-rest..."; sleep 3; done
if curl -s -u rest-admin:test "$dmn" | grep -q '"name":"diploma-eligibility.dmn"'; then
echo "diploma-eligibility DMN already deployed; skip"
else
curl -sf -u rest-admin:test -F 'file=@/work/diploma-eligibility.dmn;filename=diploma-eligibility.dmn' "$dmn" >/dev/null && echo "deployed diploma-eligibility DMN"
fi
if curl -s -u rest-admin:test "$svc?name=registratie" | grep -q '"name":"registratie"'; then
echo "registratie BPMN already deployed; skip"
else
curl -sf -u rest-admin:test -F 'file=@/work/registratie.bpmn;filename=registratie.bpmn' "$svc" >/dev/null && echo "deployed registratie BPMN"
fi
files: [{ configMap: rr-fl-bpmn, mountPath: /work }]
waitFor: [flowable-rest:8080]
# ── ACL ─────────────────────────────────────────────────────────────────────
acl:
own: true
envFrom: [otel]
env:
OTEL_SERVICE_NAME: acl
# The FQDN, not `openzaak`: OpenZaak rejects a single-label host on
# zaak-create. It must be the same host the zaaktype was seeded through
# (see the seed-zaaktype job) so the URLs stay host-consistent (ADR-0009).
Acl__OpenZaak__BaseUrl: "http://openzaak.{{ .Release.Namespace }}.svc.cluster.local:8000/"
Acl__OpenZaak__ClientId: big-reference-seed
Acl__OpenZaak__Secret: insecure-dev-secret-change-me
Acl__Defaults__Bronorganisatie: "517439943"
Acl__Defaults__VerantwoordelijkeOrganisatie: "517439943"
Acl__Defaults__Vertrouwelijkheidaanduiding: openbaar
Acl__Defaults__ZaaktypeIdentificatie: BIG-REGISTRATIE
Acl__Defaults__InformatieobjecttypeOmschrijving: Diploma
# Objecten reflects the request Host into the object url it returns, and
# publishes that url to NRC — which rejects a single-label host. The FQDN
# replaces compose's `objecten.local` alias (ADR-0029).
Acl__Objecten__BaseUrl: "http://objecten.{{ .Release.Namespace }}.svc.cluster.local:8000/"
Acl__Objecten__Token: 1234567890abcdef1234567890abcdef12345678
# Short name on purpose: Objecten only accepts an objecttype URL that
# matches the one it was configured with (infra/objecten/setup_configuration
# /data.yaml → http://objecttypen:8000/api/v2/).
Acl__Objecten__ObjecttypenBaseUrl: http://objecttypen:8000/
Acl__Objecten__ObjecttypenToken: 0123456789abcdef0123456789abcdef01234567
Acl__Objecten__ObjecttypeName: RegisterRecord
ports: [{ name: http, port: 8080 }]
probe: { httpGet: { path: /health, port: 8080 }, periodSeconds: 5 }
# ── BIG Domain Service (S-05) ───────────────────────────────────────────────
domain:
own: true
envFrom: [otel]
env:
OTEL_SERVICE_NAME: domain
Flowable__BaseUrl: http://flowable-rest:8080/flowable-rest/
Flowable__Username: rest-admin
Flowable__Password: test
Acl__BaseUrl: http://acl:8080/
ports: [{ name: http, port: 8080 }]
probe: { httpGet: { path: /health, port: 8080 }, periodSeconds: 5 }
# ── BFF ─────────────────────────────────────────────────────────────────────
bff:
own: true
envFrom: [otel]
env:
OTEL_SERVICE_NAME: bff
# In-cluster authority: Keycloak's discovery document returns the pinned
# KC_HOSTNAME issuer, which is what browser tokens carry (ADR-0010).
Keycloak__Authority: http://keycloak:8080/realms/digid
Keycloak__MedewerkerAuthority: http://keycloak:8080/realms/medewerker
Downstream__Domain__BaseUrl: http://domain:8080/
Downstream__Projection__BaseUrl: http://projection-api:8080/
Downstream__Acl__BaseUrl: http://acl:8080/
ports: [{ name: http, port: 8080 }]
probe: { httpGet: { path: /health, port: 8080 }, periodSeconds: 5 }
# ── Read projection (S-06) ──────────────────────────────────────────────────
projection-db:
image: docker.io/library/postgres:16
env:
POSTGRES_USER: projection
POSTGRES_PASSWORD: projection
POSTGRES_DB: projection
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 2Gi }
probe:
exec: { command: [pg_isready, -U, projection, -d, projection] }
periodSeconds: 5
event-subscriber:
own: true
envFrom: [otel]
env:
OTEL_SERVICE_NAME: event-subscriber
ConnectionStrings__Projection: Host=projection-db;Database=projection;Username=projection;Password=projection
Acl__BaseUrl: http://acl:8080/
EventSubscriber__Webhook__AuthToken: Bearer big-reference-notifications
ports: [{ name: http, port: 8080 }]
probe: { httpGet: { path: /health, port: 8080 }, periodSeconds: 5 }
# It migrates the projection schema on start and throws if the DB is absent.
waitFor: [projection-db:5432]
projection-api:
own: true
envFrom: [otel]
env:
OTEL_SERVICE_NAME: projection-api
ConnectionStrings__Projection: Host=projection-db;Database=projection;Username=projection;Password=projection
ports: [{ name: http, port: 8080 }]
probe: { httpGet: { path: /health, port: 8080 }, periodSeconds: 5 }
waitFor: [projection-db:5432]
# ── Portals (S-08/S-09/S-12/S-15) ───────────────────────────────────────────
# Caddy serves the Angular app and reverse-proxies its endpoint group to
# http://bff:8080 — hence the Service must stay named `bff`. Caddy resolves that
# name through the system resolver, so the DNS search domains apply and no
# upstream rewriting is needed here (ADR-0034).
self-service:
own: true
ports: [{ name: http, port: 80 }]
probe: { httpGet: { path: /, port: 80 }, periodSeconds: 5 }
files:
- configMap: portal-config-digid
mountPath: /usr/share/caddy/config.json
subPath: config.json
openbaar:
own: true
ports: [{ name: http, port: 80 }]
probe: { httpGet: { path: /, port: 80 }, periodSeconds: 5 }
behandel:
own: true
ports: [{ name: http, port: 80 }]
probe: { httpGet: { path: /, port: 80 }, periodSeconds: 5 }
files:
- configMap: portal-config-medewerker
mountPath: /usr/share/caddy/config.json
subPath: config.json
beheer:
own: true
ports: [{ name: http, port: 80 }]
probe: { httpGet: { path: /, port: 80 }, periodSeconds: 5 }
files:
- configMap: portal-config-medewerker
mountPath: /usr/share/caddy/config.json
subPath: config.json
# ── Objecttypen API (S-18a) ─────────────────────────────────────────────────
objecttypen-db:
image: docker.io/library/postgres:17-alpine
env:
POSTGRES_USER: objecttypes
POSTGRES_PASSWORD: objecttypes
POSTGRES_DB: objecttypes
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 2Gi }
probe:
exec: { command: [pg_isready, -U, objecttypes] }
periodSeconds: 5
objecttypen-redis:
image: docker.io/library/redis:7
ports: [{ name: redis, port: 6379 }]
probe: { tcpSocket: { port: 6379 } }
objecttypen:
image: docker.io/maykinmedia/objecttypes-api:3.4.2
# setup_configuration first, then the server — in ONE container, on purpose.
# Both /setup_configuration.sh and /start.sh run `manage.py migrate`, so a
# separate init Job (as compose has, ordered by depends_on) races this pod for
# the same database and Django fails with "relation already exists".
args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]
envFrom: [objecttypen]
ports: [{ name: http, port: 8000 }]
probe:
httpGet: { path: /admin/, port: 8000 }
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 30
files: [{ configMap: rr-objecttypen-config, mountPath: /app/setup_configuration }]
waitFor: [objecttypen-db:5432, objecttypen-redis:6379]
# The RegisterRecord objecttype + published version, over the API (S-18c,
# ADR-0020/ADR-0027). The uuid is pinned — Objecten identifies it by uuid.
registerrecord-init:
job: true
image: docker.io/library/python:3-slim
args: [python, /config/register.py]
env:
OBJECTTYPEN: http://objecttypen:8000
OBJECTTYPEN_TOKEN: 0123456789abcdef0123456789abcdef01234567
SCHEMA: /config/registerrecord.schema.json
files: [{ configMap: rr-registerrecord-config, mountPath: /config }]
waitFor: [objecttypen:8000]
# ── Objecten API (S-18b) ────────────────────────────────────────────────────
objecten-db:
image: docker.io/postgis/postgis:17-3.5
env:
POSTGRES_USER: objects
POSTGRES_PASSWORD: objects
POSTGRES_DB: objects
ports: [{ name: postgres, port: 5432 }]
data: { mountPath: /var/lib/postgresql/data, size: 2Gi }
probe:
exec: { command: [pg_isready, -U, objects] }
periodSeconds: 5
objecten-redis:
image: docker.io/library/redis:7
ports: [{ name: redis, port: 6379 }]
probe: { tcpSocket: { port: 6379 } }
objecten:
image: docker.io/maykinmedia/objects-api:3.4.0
# setup_configuration first, then the server — in ONE container, on purpose.
# Both /setup_configuration.sh and /start.sh run `manage.py migrate`, so a
# separate init Job (as compose has, ordered by depends_on) races this pod for
# the same database and Django fails with "relation already exists".
args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]
envFrom: [objecten]
ports: [{ name: http, port: 8000 }]
probe:
httpGet: { path: /admin/, port: 8000 }
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 30
files: [{ configMap: rr-objecten-config, mountPath: /app/setup_configuration }]
waitFor: [objecten-db:5432, objecten-redis:6379, objecttypen:8000]
# Delivers Objecten's notifications to NRC; without it every register write is
# silently undelivered (ADR-0029).
objecten-celery:
image: docker.io/maykinmedia/objects-api:3.4.0
args: [/celery_worker.sh]
envFrom: [objecten]
waitFor: [objecten-db:5432, objecten-redis:6379]
# ── Bootstrap the flow, like the local compose stack does (S-B04, ADR-0020) ──
# Seeds + publishes the BIG zaaktype through the same FQDN the ACL uses, so the
# server-assigned URLs are host-consistent. The ACL then resolves them by
# identificatie (S-27, ADR-0021) — nothing is injected back.
# Publishing validates the resultaattype against the external Selectielijst
# API, so the node needs outbound internet for this one job (ADR-0006).
seed-zaaktype:
job: true
image: docker.io/library/python:3-slim
args: [python, /seed/seed_catalogus.py]
env:
OZ_BASE: "http://openzaak.{{ .Release.Namespace }}.svc.cluster.local:8000"
OZ_PUBLISH: "1"
files: [{ configMap: rr-seed-scripts, mountPath: /seed }]
waitFor: [openzaak:8000]
# Registers the NRC abonnement on the `objecten` kanaal pointing at the
# event-subscriber, so register writes reach the projection (ADR-0030).
# Without it the openbaar register stays empty. Restart-safe and idempotent.
nrc-subscribe:
job: true
image: docker.io/library/python:3-slim
args: [python, /seed/register-abonnement.py]
env:
NRC_BASE: http://nrc-web:8000
# The script resolves this to an address for the callback URL; the FQDN
# resolves to the Service's (stable) ClusterIP, which NRC's URLValidator
# accepts — the compose stack uses the container IP for the same reason.
SINK_HOST: "event-subscriber.{{ .Release.Namespace }}.svc.cluster.local"
SINK_PORT: "8080"
SINK_AUTH: Bearer big-reference-notifications
files: [{ configMap: rr-seed-scripts, mountPath: /seed }]
waitFor: [nrc-web:8000, event-subscriber:8080]
# ── Observability backplane (S-16a, ADR-0023) ───────────────────────────────
# Off by default: these are built images too (config baked in), so switching
# them on also means pushing three more images. Enable all three together.
tempo:
enabled: false
own: true
args: ["-config.file=/etc/tempo.yaml"]
ports: [{ name: otlp, port: 4317 }, { name: http, port: 3200 }]
prometheus:
enabled: false
own: true
ports: [{ name: http, port: 9090 }]
grafana:
enabled: false
own: true
env:
GF_SECURITY_ADMIN_USER: admin
GF_SECURITY_ADMIN_PASSWORD: admin
GF_AUTH_ANONYMOUS_ENABLED: "true"
ports: [{ name: http, port: 3000 }]
+116
View File
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""Fail when the compose stack and the Helm chart stop describing the same stack.
`infra/docker-compose.yml` is CI-canonical; `infra/helm/big-reference` is a
transcription of it (ADR-0033), and until now nothing kept the two in step an
upstream image bump or a new service applied to only one of them landed
unnoticed. This compares what each side actually *deploys*, not the two files:
the rendered chart against `docker compose config`. Both tools are already
prerequisites of the `k8s-*` make targets.
Run it with `make k8s-drift`. No cluster needed.
ponytail: names and images only, as sets no per-workload env/ports/volumes.
Those differ by design in four documented places (ADR-0033), so comparing them
would mean re-encoding every deviation field by field; a tag bump and a missing
service are the drift that actually bites.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
COMPOSE = ROOT / "infra/docker-compose.yml"
CHART = ROOT / "infra/helm/big-reference"
# The busybox init container that every `waitFor` workload gets exists only in
# the chart (compose has `depends_on`). Rendering it under a sentinel makes it
# filterable without teaching the check what busybox is.
BUSYBOX = "drift-check-ignored-init-image"
# Differences that Kubernetes forces, not drift (ADR-0033). A name listed here is
# expected to be on exactly one side; anything else fails.
DEVIATIONS = {
# The four Django services apply their own setup_configuration in the web pod
# (`args: [sh, -c, "/setup_configuration.sh && exec /start.sh"]`) rather than in a
# separate init Job. Both that script and /start.sh run `manage.py migrate`, and
# Kubernetes has no `depends_on: service_completed_successfully` to serialise them,
# so the Job and its web pod migrated the same database concurrently.
"oz-init": "folded into the openzaak pod",
"nrc-init": "folded into the nrc-web pod",
"objecttypen-init": "folded into the objecttypen pod",
"objecten-init": "folded into the objecten pod",
# Compose seeds these from the host — the verify scripts `docker cp` the two
# scripts into a running container, and docker-compose.local.yml carries
# `local-seed` + `nrc-subscribe` for `make local`. A cluster has no host to seed
# from, so both became Jobs in the chart.
"seed-zaaktype": "compose seeds the catalogus from the host (infra/openzaak/seed_catalogus.py)",
"nrc-subscribe": "compose registers the abonnement from the host (infra/local/register-abonnement.py)",
}
# Workloads the observability backplane adds. Off by default in both stacks'
# defaults, so they are rendered on purpose here — otherwise their images drift
# unwatched.
OBSERVABILITY = ["tempo", "prometheus", "grafana"]
def compose_services() -> dict[str, str]:
"""Service name -> image, with ${TAG:-default} interpolation already applied."""
out = run(["docker", "compose", "-f", str(COMPOSE), "config", "--format", "json"])
return {name: svc.get("image", "") for name, svc in json.loads(out)["services"].items()}
def chart_workloads() -> dict[str, str]:
"""Workload name -> image, read back out of the rendered manifests."""
out = run(
["helm", "template", "big", str(CHART), "-n", "big", "--set", f"images.busybox={BUSYBOX}"]
+ [f"--set=workloads.{w}.enabled=true" for w in OBSERVABILITY]
)
workloads = {}
for doc in out.split("\n---"):
if not re.search(r"^kind: (Deployment|Job)$", doc, re.M):
continue
name = re.search(r"^ name: (\S+)$", doc, re.M)[1]
images = [i for i in re.findall(r"^\s+image: (\S+)$", doc, re.M) if i != BUSYBOX]
workloads[name] = images[0]
return workloads
def run(argv: list[str]) -> str:
proc = subprocess.run(argv, capture_output=True, text=True)
if proc.returncode != 0:
sys.exit(f"{argv[0]} failed:\n{proc.stderr}")
return proc.stdout
def main() -> int:
compose, chart = compose_services(), chart_workloads()
problems = []
for name in sorted(set(compose) - set(chart) - set(DEVIATIONS)):
problems.append(f" {name}: in docker-compose.yml, not in the chart")
for name in sorted(set(chart) - set(compose) - set(DEVIATIONS)):
problems.append(f" {name}: in the chart, not in docker-compose.yml")
for name in sorted(set(compose) & set(chart)):
if compose[name] != chart[name]:
problems.append(f" {name}: compose runs {compose[name]}, the chart runs {chart[name]}")
if problems:
print("compose and the Helm chart describe different stacks:\n" + "\n".join(problems))
print(
"\nPort the change to the other stack, or — if the difference is forced by\n"
"Kubernetes — declare it in DEVIATIONS in this file, with the reason."
)
return 1
print(f"no drift: {len(chart)} workloads, images identical on both stacks")
for name, why in sorted(DEVIATIONS.items()):
print(f" deviation (declared): {name}{why}")
return 0
if __name__ == "__main__":
sys.exit(main())
+60
View File
@@ -0,0 +1,60 @@
# Throwaway in-cluster OCI registry, published on NodePort 30500.
#
# Talos has no Docker daemon and no way to side-load an image, so the images built
# from this repo must come from a registry. This one lives *inside* the cluster on
# purpose: a registry on the laptop needs an inbound port opened on firewalld's
# libvirt zone (root), while pushing from the laptop to the node is outbound and
# always allowed. The node then pulls from its own NodePort.
#
# Talos must be told it speaks plain HTTP — see the machine.registries.mirrors
# patch in docs/runbooks/kubernetes-talos.md. Storage is emptyDir: if this pod is
# replaced, re-run `make k8s-images`.
apiVersion: v1
kind: Namespace
metadata:
name: registry
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry
namespace: registry
spec:
replicas: 1
strategy: { type: Recreate }
selector:
matchLabels: { app: registry }
template:
metadata:
labels: { app: registry }
spec:
containers:
- name: registry
image: docker.io/library/registry:2
env:
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
ports:
- containerPort: 5000
readinessProbe:
httpGet: { path: /v2/, port: 5000 }
volumeMounts:
- name: data
mountPath: /var/lib/registry
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: registry
namespace: registry
spec:
type: NodePort
selector: { app: registry }
ports:
- name: http
port: 5000
targetPort: 5000
nodePort: 30500
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
#
# Turn the repo's config inputs into the ConfigMaps the Helm chart mounts.
#
# This is the Kubernetes sibling of infra/seed-config.sh: the upstream Common
# Ground images are used verbatim and read their config from a mounted directory,
# so the config has to be handed to the platform out-of-band. Compose gets it via
# `docker cp` into external volumes; Kubernetes gets it as ConfigMaps created from
# the files that already live in this repo. Copying those files into the chart
# would fork them from the compose stack, so we don't.
#
# Idempotent: re-run after editing any data.yaml, then `make k8s-reseed`.
#
# Usage: seed-configmaps.sh [namespace] (default: big)
set -euo pipefail
ns="${1:-big}"
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo="$(cd "$here/../.." && pwd)"
kubectl get namespace "$ns" >/dev/null 2>&1 || kubectl create namespace "$ns"
seed() { # name <kubectl --from-file args...>
local name="$1"; shift
kubectl create configmap "$name" -n "$ns" "$@" \
--dry-run=client -o yaml | kubectl apply -f - >/dev/null
echo " seeded configmap/$name"
}
seed rr-oz-config --from-file="$repo/infra/openzaak/setup_configuration/"
seed rr-nrc-config --from-file="$repo/infra/opennotificaties/setup_configuration/"
seed rr-kc-realms --from-file="$repo/infra/keycloak/realms/"
seed rr-objecttypen-config --from-file="$repo/infra/objecttypen/setup_configuration/"
seed rr-objecten-config --from-file="$repo/infra/objecten/setup_configuration/"
# register.py + the RegisterRecord JSON schema (the __pycache__ dir is skipped:
# kubectl only takes regular files from a --from-file directory).
seed rr-registerrecord-config --from-file="$repo/infra/objecttypen-registerrecord/"
# The BPMN and the DMN are two separate Flowable deployments (S-13, ADR-0016).
seed rr-fl-bpmn \
--from-file="$repo/workflows/registratie.bpmn" \
--from-file="$repo/workflows/diploma-eligibility.dmn"
# The two bootstrap scripts the compose local stack runs as init containers
# (S-B04, ADR-0020). Stdlib-only, so a plain python image can run them.
seed rr-seed-scripts \
--from-file="$repo/infra/openzaak/seed_catalogus.py" \
--from-file="$repo/infra/local/register-abonnement.py"
+20
View File
@@ -0,0 +1,20 @@
# Overlay: make the CI compose stack usable from a HOST browser.
# Same two mechanisms infra/docker-compose.local.yml already uses — pin Keycloak's issuer to the
# host-published address, and point each portal's runtime config.json at it. The BFF needs no
# change: it discovers metadata over keycloak:8080 and the discovered issuer is the pinned
# localhost:8180, which is what browser tokens carry.
services:
keycloak:
environment:
KC_HOSTNAME: http://localhost:8180
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
self-service:
volumes:
- ./local-config/self-service.config.json:/usr/share/caddy/config.json:ro,z
behandel:
volumes:
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
# beheer is the same medewerker realm as behandel, so it reuses behandel's config verbatim.
beheer:
volumes:
- ./local-config/behandel.config.json:/usr/share/caddy/config.json:ro,z
+46 -12
View File
@@ -1,19 +1,25 @@
#!/usr/bin/env python3
"""Smoke-check the Keycloak realms: each realm's OIDC login works (password grant)
and returns its expected identifying claim. Stdlib only. Exits non-zero on failure.
and returns its expected identifying claim. The medewerker realm additionally enforces
MFA (S-15c), so its login must be refused without a TOTP code. Stdlib only.
Exits non-zero on failure.
"""
import base64, json, sys, urllib.error, urllib.parse, urllib.request
import base64, hashlib, hmac, json, struct, sys, time, urllib.error, urllib.parse, urllib.request
BASE = "http://localhost:8180"
CLIENT = "big-portal"
PWD = "test123"
# realm, user, claim ("__roles__" => check realm_access.roles), expected-contains
# Fixture TOTP secret seeded into every medewerker in infra/keycloak/realms/medewerker-realm.json.
# Keycloak HMACs the raw secret bytes, so no base32 decoding is involved.
OTP_SECRET = b"BIGMEDEWERKEROTPSEED"
# realm, user, claim ("__roles__" => check realm_access.roles), expected-contains, mfa-enforced
CHECKS = [
("digid", "jan-burger", "bsn", "123456782"),
("eherkenning", "acme-ondernemer", "kvk", "12345678"),
("eidas", "pierre-dupont", "eidas_id", "FR/NL"),
("medewerker", "merel-behandelaar", "__roles__", "behandelaar"),
("digid", "jan-burger", "bsn", "123456782", False),
("eherkenning", "acme-ondernemer", "kvk", "12345678", False),
("eidas", "pierre-dupont", "eidas_id", "FR/NL", False),
("medewerker", "merel-behandelaar", "__roles__", "behandelaar", True),
]
@@ -23,10 +29,17 @@ def decode(jwt):
return json.loads(base64.urlsafe_b64decode(p))
def grant(realm, user):
def totp(secret=OTP_SECRET, period=30, digits=6):
"""RFC 6238 code: HMAC-SHA1 over the 30-second counter, dynamically truncated."""
mac = hmac.new(secret, struct.pack(">Q", int(time.time()) // period), hashlib.sha1).digest()
o = mac[-1] & 0x0F
return str((struct.unpack(">I", mac[o:o + 4])[0] & 0x7FFFFFFF) % 10 ** digits).zfill(digits)
def grant(realm, user, **extra):
data = urllib.parse.urlencode({
"grant_type": "password", "client_id": CLIENT,
"username": user, "password": PWD, "scope": "openid",
"username": user, "password": PWD, "scope": "openid", **extra,
}).encode()
req = urllib.request.Request(
f"{BASE}/realms/{realm}/protocol/openid-connect/token", data=data,
@@ -35,11 +48,27 @@ def grant(realm, user):
return json.loads(r.read())
def second_factor_refused(realm, user):
"""The password alone must not yield a token on an MFA-enforced realm."""
try:
grant(realm, user)
except urllib.error.HTTPError as e:
return e.code in (400, 401)
return False
def main():
ok = True
for realm, user, claim, expect in CHECKS:
for realm, user, claim, expect, mfa in CHECKS:
extra = {}
if mfa:
refused = second_factor_refused(realm, user)
ok = ok and refused
print(f"{realm:12} {user:18} password-only login refused "
f"[{'OK' if refused else 'MFA NOT ENFORCED'}]")
extra = {"totp": totp()}
try:
at = decode(grant(realm, user)["access_token"])
at = decode(grant(realm, user, **extra)["access_token"])
if claim == "__roles__":
val = at.get("realm_access", {}).get("roles", [])
good = expect in val
@@ -57,4 +86,9 @@ def main():
if __name__ == "__main__":
main()
# `check_realms.py otp` prints a current code for the fixture secret — what a human demoing
# the medewerker portals types at Keycloak's OTP prompt (docs/runbooks/keycloak.md).
if len(sys.argv) > 1 and sys.argv[1] == "otp":
print(totp())
else:
main()
+37 -3
View File
@@ -2,6 +2,16 @@
"realm": "medewerker",
"enabled": true,
"displayName": "Medewerkers",
"requiredActions": [
{
"alias": "CONFIGURE_TOTP",
"name": "Configure OTP",
"providerId": "CONFIGURE_TOTP",
"enabled": true,
"defaultAction": true,
"priority": 10
}
],
"roles": {
"realm": [
{ "name": "behandelaar", "description": "Behandelt registratieaanvragen" },
@@ -43,7 +53,15 @@
"lastName": "Behandelaar",
"email": "merel@big.example.nl",
"emailVerified": true,
"credentials": [{ "type": "password", "value": "test123", "temporary": false }],
"credentials": [
{ "type": "password", "value": "test123", "temporary": false },
{
"type": "otp",
"userLabel": "seeded TOTP (fixture)",
"secretData": "{\"value\":\"BIGMEDEWERKEROTPSEED\"}",
"credentialData": "{\"subType\":\"totp\",\"digits\":6,\"counter\":0,\"period\":30,\"algorithm\":\"HmacSHA1\"}"
}
],
"realmRoles": ["behandelaar"]
},
{
@@ -53,7 +71,15 @@
"lastName": "Teamlead",
"email": "tom@big.example.nl",
"emailVerified": true,
"credentials": [{ "type": "password", "value": "test123", "temporary": false }],
"credentials": [
{ "type": "password", "value": "test123", "temporary": false },
{
"type": "otp",
"userLabel": "seeded TOTP (fixture)",
"secretData": "{\"value\":\"BIGMEDEWERKEROTPSEED\"}",
"credentialData": "{\"subType\":\"totp\",\"digits\":6,\"counter\":0,\"period\":30,\"algorithm\":\"HmacSHA1\"}"
}
],
"realmRoles": ["behandelaar", "teamlead"]
},
{
@@ -63,7 +89,15 @@
"lastName": "Beheerder",
"email": "bram@big.example.nl",
"emailVerified": true,
"credentials": [{ "type": "password", "value": "test123", "temporary": false }],
"credentials": [
{ "type": "password", "value": "test123", "temporary": false },
{
"type": "otp",
"userLabel": "seeded TOTP (fixture)",
"secretData": "{\"value\":\"BIGMEDEWERKEROTPSEED\"}",
"credentialData": "{\"subType\":\"totp\",\"digits\":6,\"counter\":0,\"period\":30,\"algorithm\":\"HmacSHA1\"}"
}
],
"realmRoles": ["beheerder"]
}
]
+11 -6
View File
@@ -2,10 +2,10 @@
"""Local-stack bootstrap (S-B04, #110, ADR-0020) — register the NRC abonnement.
Runs as the `nrc-subscribe` init container of infra/docker-compose.local.yml. Registers an
abonnement on the `zaken` kanaal pointing at the event-subscriber's /notifications callback, so
OpenZaak's notifications (zaak create + status set) reach the projection — without this the openbaar
(public) register stays empty. This is what infra/verify-notification-driver.py does for CI (minus
the test zaak it also creates).
abonnement on the `objecten` kanaal pointing at the event-subscriber's /notifications callback, so
the register writes the ACL makes (INGEDIEND on submit, INGESCHREVEN on approval) reach the
projection without this the openbaar (public) register stays empty. Since S-19b-2 the projection
is sourced from the register in Objecten, not from ZGW zaak events (ADR-0030).
The callback host is the event-subscriber's resolved **container IP**, not `event-subscriber`, because
NRC validates callbackUrl with Django's URLValidator (a single-label host is rejected — same reason the
@@ -22,6 +22,8 @@ SINK_PORT = os.environ.get("SINK_PORT", "8080")
SINK_AUTH = os.environ.get("SINK_AUTH", "Bearer big-reference-notifications")
CID = os.environ.get("OZ_CLIENT_ID", "big-reference-seed")
SECRET = os.environ.get("OZ_SECRET", "insecure-dev-secret-change-me")
# The projection is sourced from the register in Objecten, not from ZGW zaak events (S-19b-2).
KANAAL = "objecten"
def token():
@@ -60,7 +62,10 @@ def main():
status, body = call("GET", f"{NRC}/api/v1/abonnement")
for ab in (body or []) if status == 200 else []:
if str(ab.get("callbackUrl", "")).endswith("/notifications"):
if ab.get("callbackUrl") == callback:
# The kanaal is part of "current": an abonnement left over from before S-19b-2 points at
# the right callback but listens on `zaken`, and would never be replaced on IP alone.
kanalen = [k.get("naam") for k in ab.get("kanalen", [])]
if ab.get("callbackUrl") == callback and kanalen == [KANAAL]:
print(f"abonnement already current: {ab['url']}")
return
call("DELETE", ab["url"])
@@ -68,7 +73,7 @@ def main():
status, ab = call("POST", f"{NRC}/api/v1/abonnement", {
"callbackUrl": callback, "auth": SINK_AUTH,
"kanalen": [{"naam": "zaken", "filters": {}}]})
"kanalen": [{"naam": KANAAL, "filters": {}}]})
if status != 201:
sys.exit(f"create abonnement -> {status}: {json.dumps(ab)}")
print(f"abonnement registered: {ab['url']} -> {callback}")
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""S-18b (#140): prove the Objecten API is up and its static token authenticates.
Assert an unauthenticated call to /api/v2/objects is 401 and an authenticated one (the seeded dev
token) is 200 i.e. the service migrated, booted, and setup_configuration provisioned the token
and the Objecttypen service it trusts. Stdlib only so it runs in a bare python:3-slim container on
the compose network.
"""
import os
import sys
import time
import urllib.error
import urllib.request
BASE = os.environ["OBJECTEN"] # http://<ip>:8000
TOKEN = os.environ["OBJECTEN_TOKEN"]
TIMEOUT = int(os.environ.get("OBJECTEN_TIMEOUT", "60"))
def status(url, token=None):
req = urllib.request.Request(url)
if token:
req.add_header("Authorization", f"Token {token}")
try:
with urllib.request.urlopen(req, timeout=10) as r:
return r.status
except urllib.error.HTTPError as e:
return e.code
except Exception:
return 0
def main():
url = f"{BASE}/api/v2/objects"
deadline = time.time() + TIMEOUT
while time.time() < deadline:
unauth = status(url)
authed = status(url, TOKEN)
if unauth == 401 and authed == 200:
print(f"OK — {url}: no-auth {unauth}, token {authed}")
return 0
time.sleep(3)
print(f"FAIL — {url}: expected no-auth 401 + token 200, got {status(url)} / {status(url, TOKEN)}",
file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
"""S-19b-1 (#152): driver for the Objecten → NRC notification check.
Registers an abonnement on the `objecten` kanaal pointing at the webhook sink, then writes a
RegisterRecord object exactly as the ACL's ObjectenGateway does (S-19a). The caller
(run-objecten-notifications-check.sh) watches the sink for the delivery this only sets it up,
and prints `OBJECT_URL <url>` for the caller to grep on.
Delivery exercises the whole chain: Objecten its celery worker NRC nrc-beat the callback.
Anything missing (broker, worker, kanaal, notifications config) shows up as a non-delivery.
Stdlib only so it runs in a bare python:3-slim container on the compose network.
"""
import base64
import hashlib
import hmac
import json
import os
import sys
import time
import urllib.error
import urllib.request
OBJECTEN = os.environ["OBJECTEN"] # http://objecten:8000
OBJECTEN_TOKEN = os.environ["OBJECTEN_TOKEN"]
OBJECTTYPEN = os.environ["OBJECTTYPEN"] # http://objecttypen:8000
OBJECTTYPEN_TOKEN = os.environ["OBJECTTYPEN_TOKEN"]
NRC_BASE = os.environ["NRC_BASE"] # http://<nrc-ip>:8000
SINK_CALLBACK = os.environ["SINK_CALLBACK"] # http://<sink-ip>:9000/
SINK_AUTH = os.environ["SINK_AUTH"]
CLIENT_ID = os.environ.get("NRC_CLIENT_ID", "big-reference-seed")
SECRET = os.environ.get("NRC_SECRET", "insecure-dev-secret-change-me")
KANAAL = "objecten"
def mint():
"""The HS256 JWT NRC expects (same shape as infra/local/register-abonnement.py)."""
def seg(d):
return base64.urlsafe_b64encode(json.dumps(d).encode()).rstrip(b"=")
payload = seg({
"iss": CLIENT_ID, "iat": int(time.time()), "client_id": CLIENT_ID,
"user_id": CLIENT_ID, "user_representation": CLIENT_ID,
})
signing_input = seg({"typ": "JWT", "alg": "HS256"}) + b"." + payload
signature = base64.urlsafe_b64encode(
hmac.new(SECRET.encode(), signing_input, hashlib.sha256).digest()).rstrip(b"=")
return (signing_input + b"." + signature).decode()
def nrc(method, url, body=None):
"""Call NRC. `url` may be a path or an absolute URL (the list returns absolute ones)."""
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(
url if url.startswith("http") else f"{NRC_BASE}{url}", data=data, method=method,
headers={"Authorization": f"Bearer {mint()}", "Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=15) as r:
return json.load(r) if r.length != 0 else {}
except urllib.error.HTTPError as e:
# The body carries the reason (e.g. an unregistered kanaal); the status alone does not.
raise SystemExit(f"FAIL — NRC {method} {url}{e.code}: {e.read().decode(errors='replace')[:400]}")
def token_api(base, token, method, path, body=None, crs=False):
data = json.dumps(body).encode() if body is not None else None
headers = {"Authorization": f"Token {token}"}
if body is not None:
headers["Content-Type"] = "application/json"
if crs:
headers["Accept-Crs"] = "EPSG:4326"
if body is not None:
headers["Content-Crs"] = "EPSG:4326"
req = urllib.request.Request(f"{base}{path}", data=data, method=method, headers=headers)
with urllib.request.urlopen(req, timeout=15) as r:
return json.load(r) if r.length != 0 else {}
def subscribe():
"""Register an abonnement on the objecten kanaal, replacing a stale one for the same callback."""
# NRC returns a bare list here, not a paginated envelope.
for existing in nrc("GET", "/api/v1/abonnement") or []:
if existing.get("callbackUrl") == SINK_CALLBACK:
nrc("DELETE", existing["url"])
nrc("POST", "/api/v1/abonnement", {
"callbackUrl": SINK_CALLBACK,
"auth": SINK_AUTH,
"kanalen": [{"naam": KANAAL, "filters": {}}],
})
print(f">> abonnement on '{KANAAL}' -> {SINK_CALLBACK}")
def objecttype_url():
results = token_api(OBJECTTYPEN, OBJECTTYPEN_TOKEN, "GET", "/api/v2/objecttypes").get("results", [])
match = next((o for o in results if o.get("name") == "RegisterRecord"), None)
if not match:
print("FAIL — no RegisterRecord objecttype in Objecttypen", file=sys.stderr)
raise SystemExit(1)
return match["url"]
def main():
subscribe()
reference = f"NOTIF-{int(time.time())}"
created = token_api(OBJECTEN, OBJECTEN_TOKEN, "POST", "/api/v2/objects", {
"type": objecttype_url(),
"record": {
"typeVersion": 1,
"data": {"id": f"zaak-{reference}", "status": "INGESCHREVEN", "reference": reference},
"startAt": time.strftime("%Y-%m-%d"),
},
}, crs=True)
print(f">> wrote RegisterRecord {created['url']}")
# An NRC notification carries no record data — only hoofdObject/resourceUrl — so the object
# URL, not the reference in its data, is what the caller can correlate the delivery on.
print(f"OBJECT_URL {created['url']}")
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,59 @@
# Objecten API setup_configuration (S-18b). Streamed into the external rr-objecten-config volume by
# infra/seed-config.sh and applied by objecten-init (RUN_SETUP_CONFIG). Declarative + idempotent.
#
# Two things: (1) register the Objecttypen API (S-18a) as a trusted service so an object can
# reference its objecttype — authenticating with the dev static token Objecttypen provisioned; and
# (2) a dev static token so peers (the ACL, S-19) can write objects here. Dev-only, not for prod.
# (1) Trust the Objecttypen API. `orc` = overige RESTful component (how zgw_consumers classifies the
# Objecttypen API). The RegisterRecord objecttype (S-18c) will reference an objecttype under this
# service by uuid.
zgw_consumers_config_enable: true
zgw_consumers:
services:
- identifier: objecttypen
label: Objecttypen API
api_type: orc
api_root: http://objecttypen:8000/api/v2/
auth_type: api_key
header_key: Authorization
header_value: Token 0123456789abcdef0123456789abcdef01234567
# (1b) The NRC Objecten publishes register-record events to (S-19b-1, ADR-0029). Same shape and
# same big-reference-seed credential OpenZaak publishes with — NRC verifies the JWT and
# authorizes it via OpenZaak's AC, which grants that client heeft_alle_autorisaties.
- identifier: nrc
label: Open Notificaties
api_type: nrc
api_root: http://nrc-web:8000/api/v1/
auth_type: zgw
client_id: big-reference-seed
secret: insecure-dev-secret-change-me
# (2) Permit the RegisterRecord objecttype (S-19a). Objecten refuses to store an object whose
# objecttype it has not been configured with ("ObjectType with url=… is not configured"), and it
# identifies one by uuid — which is why infra/objecttypen-registerrecord/register.py pins that uuid
# instead of letting Objecttypen assign one. Keep the two in step.
objecttypes_config_enable: true
objecttypes:
items:
- uuid: 1f4b4e26-8b1f-4e2f-9d6c-6a1b7a2f0e01
name: RegisterRecord
service_identifier: objecttypen
# (3) Static API token peers use to write/read objects.
tokenauth_config_enable: true
tokenauth:
items:
- identifier: register-referentie
token: 1234567890abcdef1234567890abcdef12345678
contact_person: Register Referentie
email: admin@localhost
organization: Respellion
is_superuser: true
# (4) Point Objecten's notifications at that NRC service (S-19b-1, ADR-0029). Requires
# NOTIFICATIONS_DISABLED=false plus a celery broker + worker — without the worker the message is
# queued and never sent, which is exactly the half-wired state S-19a refused to ship (ADR-0028).
notifications_config_enable: true
notifications_config:
notifications_api_service_identifier: nrc
@@ -0,0 +1,77 @@
#!/usr/bin/env python3
"""S-18c (#141): register the RegisterRecord objecttype + a published version in the Objecttypen API.
Run by the `registerrecord-init` compose one-shot once Objecttypen is healthy. The Objecttypen API's
setup_configuration (3.4.2) can only provision tokens it has no declarative objecttype step so
the objecttype is created over the API here (the ADR-0020 self-seed pattern), idempotently: if a
"RegisterRecord" objecttype with a published version already exists, it's a no-op. Stdlib only.
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
BASE = os.environ.get("OBJECTTYPEN", "http://objecttypen:8000").rstrip("/")
TOKEN = os.environ["OBJECTTYPEN_TOKEN"]
SCHEMA_PATH = os.environ.get("SCHEMA", "/config/registerrecord.schema.json")
NAME = "RegisterRecord"
# Pinned rather than server-assigned (S-19a): the Objecten API will only accept objects whose
# objecttype it has been configured with *by uuid*, and its own setup_configuration is a static
# file applied before this one-shot runs. A fixed uuid lets both sides be declared up front instead
# of threading a seed-time value between two containers. See infra/objecten/setup_configuration.
UUID = "1f4b4e26-8b1f-4e2f-9d6c-6a1b7a2f0e01"
def api(method, path, body=None):
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(
f"{BASE}{path}", data=data, method=method,
headers={"Authorization": f"Token {TOKEN}", "Content-Type": "application/json"},
)
with urllib.request.urlopen(req, timeout=15) as r:
return json.load(r) if r.length != 0 else {}
def wait_ready():
"""Objecttypen depends_on health already, but tolerate a slow first request."""
for _ in range(20):
try:
api("GET", "/api/v2/objecttypes")
return
except (urllib.error.URLError, ConnectionError, TimeoutError):
time.sleep(3)
api("GET", "/api/v2/objecttypes") # last try, let it raise
def main():
schema = json.load(open(SCHEMA_PATH))
wait_ready()
existing = next(
(o for o in api("GET", "/api/v2/objecttypes").get("results", []) if o.get("name") == NAME),
None,
)
if existing and existing.get("versions"):
print(f"RegisterRecord already registered ({len(existing['versions'])} version(s)) — no-op")
return 0
ot = existing or api("POST", "/api/v2/objecttypes", {
"uuid": UUID,
"name": NAME,
"namePlural": "RegisterRecords",
"description": schema.get("description", ""),
"dataClassification": "open", # public-safe: the openbaar register may show it
})
uuid = ot["uuid"]
ver = api("POST", f"/api/v2/objecttypes/{uuid}/versions", {
"status": "published",
"jsonSchema": schema,
})
print(f"registered RegisterRecord {uuid} v{ver.get('version')} ({ver.get('status')})")
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,23 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "RegisterRecord",
"description": "Public-safe register entry shown in the openbaar (public) register. Mirrors the BFF's OpenbaarEntry (services/bff/Bff.Api/DownstreamClients.cs) — deliberately NO bsn or naam. S-19 (#20) writes records against this schema in the Objecten API on approval. See ADR-0027.",
"type": "object",
"additionalProperties": false,
"required": ["id", "status"],
"properties": {
"id": {
"type": "string",
"description": "Zaak id — the register entry's stable primary key (the projection key)."
},
"status": {
"type": "string",
"enum": ["INGEDIEND", "INGESCHREVEN"],
"description": "Registration lifecycle status (RegistrationStatus)."
},
"reference": {
"type": ["string", "null"],
"description": "Citizen-facing zaak identificatie shown publicly (ADR-0012)."
}
}
}
+12
View File
@@ -25,3 +25,15 @@ storage:
path: /var/tempo/blocks
wal:
path: /var/tempo/wal
# #156: don't let the distributor evict its own ingester. Tempo runs single-binary here, so the
# distributor and the ingester are the same process and the "pool" holds exactly one, in-process,
# member. dskit still health-checks it over loopback gRPC with a 1s deadline (checkinterval 15s);
# on the shared CI runner a transient stall blows that deadline, the only ingester is dropped from
# the pool ("removing distributor_pool failing healthcheck"), and every push then fails ("pusher
# failed to consume trace data", err="context canceled") until the next check — silently losing
# spans, which is how verify-tracing flaked. With one in-process ingester the check can never route
# around a failure, so it can only ever discard data. Turn it off.
ingester_client:
pool_config:
healthcheckenabled: false
@@ -29,7 +29,9 @@ autorisaties_api_config_enable: true
autorisaties_api:
authorizations_api_service_identifier: openzaak-ac
# 4. The kanaal OpenZaak publishes zaak events on.
# 4. The kanalen publishers announce on: `zaken` (OpenZaak) and `objecten` (Objecten, S-19b-1).
# Both authenticate with the big-reference-seed credential above, which OpenZaak's AC grants
# heeft_alle_autorisaties — so no separate publisher authorization is needed for Objecten.
notifications_kanalen_config_enable: true
notifications_kanalen_config:
items:
@@ -39,3 +41,11 @@ notifications_kanalen_config:
- bronorganisatie
- zaaktype
- vertrouwelijkheidaanduiding
# 5. The kanaal Objecten publishes register-record events on (S-19b-1, ADR-0029). Its name is
# fixed by the Objects API itself (NOTIFICATIONS_KANAAL = "objecten"), not chosen here. The
# filter set matches what the Objects API sends as kenmerken, so an abonnement can narrow by
# objecttype rather than receiving every object write in the register.
- naam: objecten
documentatie_link: https://objects-and-objecttypes-api.readthedocs.io/
filters:
- object_type
+37 -5
View File
@@ -7,10 +7,34 @@ redirects it into $GITHUB_STEP_SUMMARY. Stdlib only.
"""
import json
import os
import re
import sys
STATUS_ICON = {"expected": "", "unexpected": "", "skipped": "⏭️", "flaky": "⚠️"}
# A verdict alone still costs a log dive, and a killed or truncated job leaves no log to dive into
# (#161) — so a failing spec carries its first error into the table. Playwright errors are multi-line
# with a "Call log:", which a markdown table cell cannot hold, so they are flattened and clipped.
ERROR_CLIP = 300
def first_error(spec):
"""The first error message across a spec's test results, flattened for one table cell."""
for test in spec.get("tests", []):
for result in test.get("results", []):
for error in result.get("errors", []):
message = (error.get("message") or "").strip()
if not message:
continue
# Strip ANSI colour, collapse to one line, and keep it inside the cell.
message = re.sub(r"\x1b\[[0-9;]*m", "", message)
message = " ".join(message.split())
if len(message) > ERROR_CLIP:
message = message[:ERROR_CLIP - 1].rstrip() + ""
# `|` would end the cell early.
return message.replace("|", "\\|")
return ""
def walk(suite, out):
for spec in suite.get("specs", []):
@@ -22,7 +46,8 @@ def walk(suite, out):
else "expected" if spec.get("ok", False)
else "unexpected")
out.append({"file": spec.get("file") or suite.get("file") or suite.get("title", ""),
"title": spec.get("title", ""), "status": status})
"title": spec.get("title", ""), "status": status,
"error": first_error(spec) if status in ("unexpected", "flaky") else ""})
for child in suite.get("suites", []):
walk(child, out)
@@ -46,10 +71,17 @@ def main(path):
if not specs:
print("_No specs ran._")
return 0
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
# The failure column only earns its width when something failed.
if any(s["error"] for s in specs):
print("| Spec | Result | Why |")
print("| ---- | :----: | --- |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} | {s['error']} |")
else:
print("| Spec | Result |")
print("| ---- | :----: |")
for s in specs:
print(f"| {s['file']} {s['title']} | {STATUS_ICON.get(s['status'], '')} |")
return 0
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""S-18c (#141): prove the RegisterRecord objecttype is registered + published in the Objecttypen API.
Assert the objecttype named "RegisterRecord" exists, has a **published** version, and that version's
jsonSchema carries the public-safe fields (id, status, reference) i.e. registerrecord-init ran and
seeded the schema S-19 will write records against. Stdlib only so it runs in a bare python:3-slim
container on the compose network.
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
BASE = os.environ["OBJECTTYPEN"] # http://<ip>:8000
TOKEN = os.environ["OBJECTTYPEN_TOKEN"]
TIMEOUT = int(os.environ.get("REGISTERRECORD_TIMEOUT", "60"))
NAME = "RegisterRecord"
EXPECTED_FIELDS = {"id", "status", "reference"}
def get(path):
req = urllib.request.Request(f"{BASE}{path}", headers={"Authorization": f"Token {TOKEN}"})
with urllib.request.urlopen(req, timeout=10) as r:
return json.load(r)
def check():
"""Return (ok, detail). Raises on transport errors so the caller can retry."""
ots = get("/api/v2/objecttypes").get("results", [])
match = next((o for o in ots if o.get("name") == NAME), None)
if not match:
return False, f"no objecttype named {NAME!r} (have: {[o.get('name') for o in ots]})"
if not match.get("versions"):
return False, f"{NAME} exists but has no versions"
# The versions list holds URLs; fetch each to find a published one.
for ver_url in match["versions"]:
ver = get(ver_url[len(BASE):] if ver_url.startswith(BASE) else ver_url)
if ver.get("status") != "published":
continue
props = set((ver.get("jsonSchema") or {}).get("properties", {}))
if not EXPECTED_FIELDS <= props:
return False, f"published version missing fields: {EXPECTED_FIELDS - props}"
return True, f"{NAME} v{ver.get('version')} published, fields={sorted(props)}"
return False, f"{NAME} has versions but none are published"
def main():
deadline = time.time() + TIMEOUT
detail = "no attempt"
while time.time() < deadline:
try:
ok, detail = check()
if ok:
print(f"OK — {detail}")
return 0
except (urllib.error.URLError, ConnectionError, TimeoutError) as e:
detail = f"transport: {e}"
time.sleep(3)
print(f"FAIL — {detail}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -142,6 +142,7 @@ still="$(printf '%s' "$resp" | task_for_reg "$reg_id")"
[ -z "$still" ] || { echo "FAIL — Beoordelen task $still still active after completion" >&2; exit 1; }
echo "OK — behandelaar claimed and completed the Beoordelen task; the registratie process finished"
# ── S-11: withdrawal. A second registration parks at Beoordelen; the citizen withdraws it via the
# domain, which delivers the RegistratieIngetrokken message to the task's execution, tripping the
# BPMN boundary event so the process ends and the Beoordelen task disappears (ADR-0014). ────────────
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
#
# S-18b (#140): assert the Objecten API is healthy + its static token authenticates, against an
# ALREADY-RUNNING stack. Runs the check in a python:3-slim container on the stack network (the
# service is reached by container IP; the runner can't reach published ports — gitea-actions-gotchas.md
# §5/§6). Does NOT manage the stack lifecycle.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# The dev token provisioned by infra/objecten/setup_configuration/data.yaml.
TOKEN="${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}"
ot="$(docker ps -q --filter 'name=objecten' --filter 'health=healthy' | head -1)"
[ -n "$ot" ] || ot="$(docker ps -q --filter 'name=[-_]objecten[-_]' | head -1)"
[ -n "$ot" ] || { echo "ERROR: no running objecten container — bring the stack up first" >&2; exit 1; }
net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$ot" | head -1)"
ip="$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$ot")"
echo ">> network=$net objecten=$ip"
cid="$(docker create --network "$net" \
-e "OBJECTEN=http://$ip:8000" -e "OBJECTEN_TOKEN=$TOKEN" \
-e "OBJECTEN_TIMEOUT=${OBJECTEN_TIMEOUT:-60}" \
python:3-slim python /objecten-check.py)"
docker cp "$here/objecten-check.py" "$cid:/objecten-check.py" >/dev/null
rc=0; docker start -a "$cid" || rc=$?
docker rm -f "$cid" >/dev/null
exit $rc
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
#
# S-19b-1 (#152): verify the Objecten → NRC notification path against an ALREADY-RUNNING full
# stack. Registers an abonnement on the `objecten` kanaal pointing at a throwaway webhook sink,
# writes a RegisterRecord object (exactly as the ACL does on approval, S-19a), and asserts the sink
# receives the notification.
#
# This is the whole publish chain in one assertion: Objecten → its celery worker → NRC → nrc-beat →
# the subscriber callback. S-19a deliberately left it disconnected (ADR-0028); this proves it is
# connected for real, rather than merely configured.
#
# All in-network, reaching services by container IP (a single-label host isn't URL-valid for NRC's
# callbackUrl validator; the runner can't reach published ports — gitea-actions-gotchas.md §5/§6).
# EXCEPT Objecttypen, which must be reached by SERVICE NAME: it echoes the request Host into the
# objecttype `url` and Objecten only accepts the one matching its configured api_root (ADR-0028);
# and Objecten, reached by its `objecten.local` alias because it reflects the request Host into the
# notification's hoofdObject/resourceUrl, which NRC validates as a URL (ADR-0029).
#
# Does NOT manage the stack lifecycle, but cleans up the sink/driver it creates.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SINK_AUTH="Bearer objecten-notification-sink-token"
cleanup() { docker rm -f rr-osink rr-overify >/dev/null 2>&1 || true; }
trap cleanup EXIT
ip() { docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$1"; }
# Anchored on the compose replica suffix so they don't also match objecten-db / objecten-redis.
obj="$(docker ps -q --filter 'name=objecten[-_][0-9]+$' | head -1)"
nrc="$(docker ps -q --filter 'name=nrc-web' | head -1)"
[ -n "$obj" ] || { echo "ERROR: no running objecten container — bring the stack up first" >&2; exit 1; }
[ -n "$nrc" ] || { echo "ERROR: no running nrc-web container — bring the stack up first" >&2; exit 1; }
net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$obj" | head -1)"
nrc_ip="$(ip "$nrc")"
echo ">> network=$net nrc=$nrc_ip"
echo ">> starting the webhook sink"
docker rm -f rr-osink >/dev/null 2>&1 || true
sink="$(docker create --network "$net" --name rr-osink -e "EXPECTED_AUTH=$SINK_AUTH" \
python:3-slim python /sink.py)"
docker cp "$here/notification-sink.py" "$sink:/sink.py" >/dev/null
docker start "$sink" >/dev/null
sleep 1
sink_ip="$(ip rr-osink)"
echo ">> sink at $sink_ip:9000"
echo ">> registering the abonnement + writing a RegisterRecord"
docker rm -f rr-overify >/dev/null 2>&1 || true
drv="$(docker create --network "$net" --name rr-overify \
-e "OBJECTEN=http://objecten.local:8000" \
-e "OBJECTEN_TOKEN=${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}" \
-e "OBJECTTYPEN=http://objecttypen:8000" \
-e "OBJECTTYPEN_TOKEN=${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}" \
-e "NRC_BASE=http://$nrc_ip:8000" \
-e "SINK_CALLBACK=http://$sink_ip:9000/" -e "SINK_AUTH=$SINK_AUTH" \
python:3-slim python /driver.py)"
docker cp "$here/objecten-notifications-check.py" "$drv:/driver.py" >/dev/null
docker start -a "$drv"
object_url="$(docker logs rr-overify 2>/dev/null | sed -n 's/^OBJECT_URL //p' | head -1)"
docker rm -f rr-overify >/dev/null
[ -n "$object_url" ] || { echo "FAIL — the driver did not write a RegisterRecord" >&2; exit 1; }
echo ">> wrote $object_url"
# Correlate on the object URL: a notification carries hoofdObject/resourceUrl, never the record
# data, so the reference inside the record is not in the delivered message.
echo ">> waiting for the notification to reach the sink"
for _ in $(seq 1 "${NOTIFICATION_TRIES:-40}"); do
if docker logs rr-osink 2>&1 | grep -qF "$object_url"; then
echo "OK — Objecten published to NRC and the abonnement delivered it:"
docker logs rr-osink 2>&1 | grep -F "$object_url" | tail -1 | cut -c1-500
exit 0
fi
sleep 2
done
echo "FAIL — no 'objecten' notification for $object_url reached the sink." >&2
echo " Objecten accepted the write, so the gap is downstream: the celery broker/worker," >&2
echo " the kanaal registration, or Objecten's notifications_config." >&2
echo "--- sink log ---" >&2; docker logs rr-osink 2>&1 | tail -8 >&2
echo "--- objecten log ---" >&2; docker logs "$obj" 2>&1 | tail -15 >&2
exit 1
+50 -18
View File
@@ -1,18 +1,26 @@
#!/usr/bin/env bash
#
# Verify the end-to-end read-projection path (S-06) against an ALREADY-RUNNING full stack:
# OpenZaak → NRC → Event Subscriber → projection → projection-api. Seeds a published BIG
# zaaktype (idempotent), registers an abonnement on the `zaken` kanaal pointing at the real
# Event Subscriber's /notifications callback (with the bearer it enforces), creates a zaak,
# and asserts projection-api serves a row for that zaak with status INGEDIEND.
# Verify the end-to-end read-projection path (S-06, re-sourced by S-19b-2) against an ALREADY-RUNNING
# full stack: ACL → Objecten → NRC → Event Subscriber → projection → projection-api. Seeds a
# published BIG zaaktype (idempotent), registers an abonnement on the `objecten` kanaal pointing at
# the real Event Subscriber's /notifications callback (with the bearer it enforces), opens a zaak
# *through the ACL*, and asserts projection-api serves a row for it with status INGEDIEND.
#
# The zaak is opened through the ACL, not straight against OpenZaak: since ADR-0030 the projection is
# derived from the RegisterRecord in Objecten, and the ACL is what writes that record (INGEDIEND on
# submit). A zaak created behind the ACL's back produces no register write and so no projection row —
# which is the point of the re-source.
#
# All in-network, reaching services by container IP — single-label hosts aren't URL-valid and
# the runner can't reach published ports (gitea-actions-gotchas.md §5/§6). Reuses the
# notification driver to register the abonnement + create the zaak. Does NOT manage the stack
# lifecycle (the caller owns bring-up + teardown). Plain docker primitives only. See ADR-0007/0008.
# the runner can't reach published ports (gitea-actions-gotchas.md §5/§6). Does not own the stack
# lifecycle (the caller brings it up and tears it down), but does recreate the `acl` service to
# repoint it — see below, and run-domain-check.sh, which does the same. Plain docker primitives only.
# See ADR-0007/0008/0030.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
root="$(cd "$here/.." && pwd)"
compose="$root/infra/docker-compose.yml"
WEBHOOK_AUTH="${NOTIFICATION_WEBHOOK_TOKEN:-Bearer big-reference-notifications}"
cleanup() { docker rm -f rr-pverify rr-pquery >/dev/null 2>&1 || true; }
@@ -24,11 +32,13 @@ oz="$(docker ps -q --filter 'name=[-_]openzaak[-_]' | head -1)"
nrc="$(docker ps -q --filter 'name=nrc-web' | head -1)"
es="$(docker ps -q --filter 'name=event-subscriber' | head -1)"
proj="$(docker ps -q --filter 'name=projection-api' | head -1)"
acl="$(docker ps -q --filter 'name=[-_]acl[-_]' | head -1)"
[ -n "$oz" ] && [ -n "$nrc" ] || { echo "ERROR: OpenZaak and/or NRC not running — bring the stack up first" >&2; exit 1; }
[ -n "$es" ] && [ -n "$proj" ] || { echo "ERROR: event-subscriber and/or projection-api not running — bring the stack up first" >&2; exit 1; }
[ -n "$acl" ] || { echo "ERROR: acl not running — bring the stack up first" >&2; exit 1; }
net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$oz" | head -1)"
oz_ip="$(ip "$oz")"; nrc_ip="$(ip "$nrc")"; es_ip="$(ip "$es")"; proj_ip="$(ip "$proj")"
echo ">> network=$net openzaak=$oz_ip nrc=$nrc_ip event-subscriber=$es_ip projection-api=$proj_ip"
oz_ip="$(ip "$oz")"; nrc_ip="$(ip "$nrc")"; es_ip="$(ip "$es")"; proj_ip="$(ip "$proj")"; acl_ip="$(ip "$acl")"
echo ">> network=$net openzaak=$oz_ip nrc=$nrc_ip event-subscriber=$es_ip projection-api=$proj_ip acl=$acl_ip"
echo ">> seeding a published BIG zaaktype (idempotent)"
sid="$(docker create --network "$net" -e "OZ_BASE=http://$oz_ip:8000" -e OZ_PUBLISH=1 \
@@ -37,19 +47,39 @@ docker cp "$here/openzaak/seed_catalogus.py" "$sid:/seed.py" >/dev/null
docker start -a "$sid"
docker rm -f "$sid" >/dev/null
echo ">> registering abonnement at the Event Subscriber + creating a zaak"
echo ">> registering the event-subscriber abonnement on the objecten kanaal"
docker rm -f rr-pverify >/dev/null 2>&1 || true
# The same script the local stack uses (ADR-0020), so both paths register the identical abonnement.
drv="$(docker create --network "$net" --name rr-pverify \
-e "OZ_BASE=http://$oz_ip:8000" -e "NRC_BASE=http://$nrc_ip:8000" \
-e "SINK_CALLBACK=http://$es_ip:8080/notifications" -e "SINK_AUTH=$WEBHOOK_AUTH" \
python:3-slim python /driver.py)"
docker cp "$here/verify-notification-driver.py" "$drv:/driver.py" >/dev/null
-e "NRC_BASE=http://$nrc_ip:8000" \
-e "SINK_HOST=$es_ip" -e "SINK_PORT=8080" -e "SINK_AUTH=$WEBHOOK_AUTH" \
python:3-slim python /subscribe.py)"
docker cp "$here/local/register-abonnement.py" "$drv:/subscribe.py" >/dev/null
docker start -a "$drv"
zaak_url="$(docker logs rr-pverify 2>/dev/null | sed -n 's/^ZAAK_CREATED //p' | head -1)"
docker rm -f rr-pverify >/dev/null
[ -n "$zaak_url" ] || { echo "ERROR: driver did not create a zaak" >&2; exit 1; }
# OpenZaak reflects the request Host into the zaaktype `url` it returns, and then rejects that same
# URL on zaak-create when the host is single-label ("Voer een geldige URL in."). The stack's ACL is
# configured with `http://openzaak:8000/`, so it must be repointed at OpenZaak's container IP before
# it can open a zaak — exactly what run-domain-check.sh does, and the same class of constraint as the
# `objecten.local` alias (ADR-0029). The ACL resolves the zaaktype itself (S-27, ADR-0021), so the
# base URL is the only thing to inject.
echo ">> recreating the acl service pointed at OpenZaak's IP"
ACL_OPENZAAK_BASEURL="http://$oz_ip:8000/" docker compose -f "$compose" up -d acl
WAIT_TIMEOUT="${WAIT_TIMEOUT:-120}" bash "$here/wait-healthy.sh" acl
# The container is replaced, so its IP may have changed.
acl="$(docker ps -q --filter 'name=[-_]acl[-_]' | head -1)"
acl_ip="$(ip "$acl")"
echo ">> opening a zaak through the ACL (which writes the INGEDIEND register record)"
reference="PROJ-$(date +%s)"
zaak_url="$(docker run --rm --network "$net" curlimages/curl:latest \
-fsS -X POST "http://$acl_ip:8080/zaken" -H 'Content-Type: application/json' \
-d "{\"bsn\":\"123456782\",\"reference\":\"$reference\"}" \
| sed -n 's/.*"zaakUrl":"\([^"]*\)".*/\1/p')"
[ -n "$zaak_url" ] || { echo "ERROR: the ACL did not open a zaak" >&2; exit 1; }
zaak_uuid="${zaak_url##*/}"
echo ">> zaak created: $zaak_url"
echo ">> zaak created: $zaak_url (reference $reference)"
echo ">> polling projection-api for the projected row (status INGEDIEND)"
for _ in $(seq 1 30); do
@@ -63,6 +93,8 @@ for _ in $(seq 1 30); do
sleep 2
done
echo "FAIL — projection-api never served an INGEDIEND row for zaak $zaak_uuid" >&2
echo " The chain is ACL → Objecten → NRC → event-subscriber → projection (ADR-0030)." >&2
echo "--- event-subscriber log ---" >&2; docker logs "$es" 2>&1 | tail -10 >&2
echo "--- projection-api log ---" >&2; docker logs "$proj" 2>&1 | tail -10 >&2
echo "--- acl log ---" >&2; docker logs "$acl" 2>&1 | tail -10 >&2
exit 1
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
#
# S-18c (#141): assert the RegisterRecord objecttype is registered + published in the Objecttypen
# API, against an ALREADY-RUNNING stack. Runs the check in a python:3-slim container on the stack
# network (the service is reached by container IP; the runner can't reach published ports —
# gitea-actions-gotchas.md §5/§6). Does NOT manage the stack lifecycle.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# The dev token provisioned by infra/objecttypen/setup_configuration/data.yaml.
TOKEN="${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}"
ot="$(docker ps -q --filter 'name=objecttypen' --filter 'health=healthy' | head -1)"
[ -n "$ot" ] || ot="$(docker ps -q --filter 'name=[-_]objecttypen[-_]' | head -1)"
[ -n "$ot" ] || { echo "ERROR: no running objecttypen container — bring the stack up first" >&2; exit 1; }
net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$ot" | head -1)"
ip="$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$ot")"
echo ">> network=$net objecttypen=$ip"
cid="$(docker create --network "$net" \
-e "OBJECTTYPEN=http://$ip:8000" -e "OBJECTTYPEN_TOKEN=$TOKEN" \
-e "REGISTERRECORD_TIMEOUT=${REGISTERRECORD_TIMEOUT:-60}" \
python:3-slim python /registerrecord-check.py)"
docker cp "$here/registerrecord-check.py" "$cid:/registerrecord-check.py" >/dev/null
rc=0; docker start -a "$cid" || rc=$?
docker rm -f "$cid" >/dev/null
exit $rc
+4 -2
View File
@@ -13,7 +13,7 @@
# subcommand. Fixed-name `external` volumes keep the names deterministic across
# both runtimes. See docs/runbooks/gitea-actions-gotchas.md.
#
# Usage: seed-config.sh <key> [<key> ...] where key ∈ { oz, kc, fl }
# Usage: seed-config.sh <key> [<key> ...] where key ∈ { oz, nrc, kc, fl, objecttypen, objecten, registerrecord }
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
@@ -33,7 +33,7 @@ populate() { # volume source(file or dir/.)
echo " seeded $vol"
}
[ "$#" -gt 0 ] || { echo "usage: seed-config.sh <oz|nrc|kc|fl|objecttypen> ..." >&2; exit 2; }
[ "$#" -gt 0 ] || { echo "usage: seed-config.sh <oz|nrc|kc|fl|objecttypen|objecten|registerrecord> ..." >&2; exit 2; }
# The registratie process (BPMN) and its diploma-eligibility DMN are deployed as SEPARATE Flowable
# deployments — the process engine and the DMN engine each own theirs (S-13, ADR-0016). flowable-rest
@@ -50,6 +50,8 @@ for key in "$@"; do
nrc) populate rr-nrc-config "$here/opennotificaties/setup_configuration/." ;;
kc) populate rr-kc-realms "$here/keycloak/realms/." ;;
objecttypen) populate rr-objecttypen-config "$here/objecttypen/setup_configuration/." ;;
objecten) populate rr-objecten-config "$here/objecten/setup_configuration/." ;;
registerrecord) populate rr-registerrecord-config "$here/objecttypen-registerrecord/." ;;
fl) d="$(mktemp -d)"; stage_flowable_workflows "$d"; populate rr-fl-bpmn "$d/." ;;
*) echo "unknown seed key: $key" >&2; exit 2 ;;
esac
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/env python3
"""Self-check for infra/playwright-summary.py — stdlib asserts, no framework.
Run: python3 infra/test_playwright_summary.py (also runs in `make unit`).
A red e2e is only useful if the job summary says WHY it failed: #161 lost a 36-minute
verify-stack job whose only surviving output was one line with no assertion detail.
"""
import importlib.util
import io
import json
import os
import tempfile
from contextlib import redirect_stdout
# The script's filename is not a valid module name, so load it by path.
spec = importlib.util.spec_from_file_location(
"playwright_summary",
os.path.join(os.path.dirname(os.path.abspath(__file__)), "playwright-summary.py"),
)
summary = importlib.util.module_from_spec(spec)
spec.loader.exec_module(summary)
def render(report):
"""Run the renderer over a report dict and return its markdown."""
with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as fh:
json.dump(report, fh)
path = fh.name
try:
out = io.StringIO()
with redirect_stdout(out):
summary.main(path)
return out.getvalue()
finally:
os.unlink(path)
def spec_entry(title, status, errors=()):
return {
"title": title,
"file": "catalogus.spec.ts",
"ok": status == "expected",
"tests": [{"status": status, "results": [{"errors": [{"message": m} for m in errors]}]}],
}
def test_failing_spec_reports_why():
md = render({
"stats": {"expected": 4, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 108_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the published zaaktypen in the catalogus", "unexpected",
["locator.fill: Test timeout of 90000ms exceeded.\n"
"Call log:\n - waiting for locator('#username')\n"]),
]}],
})
assert "" in md, md
# The point of the slice: the summary names the cause, not just the verdict.
assert "Test timeout of 90000ms exceeded" in md, md
assert "waiting for locator('#username')" in md, md
# A multi-line Playwright error must not break out of its table row.
assert not any(line.startswith("Call log:") for line in md.splitlines()), md
def test_real_playwright_error_is_flattened():
# A real report's message is multi-line and ANSI-coloured, and embeds the source snippet with
# `|` gutters — all three would break the table cell. Shape verified against an actual
# @playwright/test 1.61 JSON report.
md = render({
"stats": {"expected": 0, "unexpected": 1, "flaky": 0, "skipped": 0, "duration": 1_000},
"suites": [{"file": "catalogus.spec.ts", "specs": [
spec_entry("a beheerder sees the catalogus", "unexpected",
["Error: expect(locator).toBeVisible() failed\n\n"
"\x1b[2mLocator: \x1b[22mgetByRole('heading')\n"
" 12 | await login(page);\n> 13 | await expect(heading).toBeVisible();\n"]),
]}],
})
row = [line for line in md.splitlines() if line.startswith("| catalogus.spec.ts")][0]
assert "\x1b" not in row, row
assert "Locator: getByRole('heading')" in row, row
# Every literal `|` from the snippet gutters is escaped, so the row keeps exactly 3 cells.
assert row.count("|") - row.count("\\|") == 4, row
def test_passing_run_stays_quiet():
md = render({
"stats": {"expected": 1, "unexpected": 0, "flaky": 0, "skipped": 0, "duration": 5_000},
"suites": [{"file": "catalogus.spec.ts",
"specs": [spec_entry("a beheerder sees the catalogus", "expected")]}],
})
assert "" in md, md
assert "timeout" not in md.lower(), md
def test_missing_report_is_not_a_crash():
out = io.StringIO()
with redirect_stdout(out):
rc = summary.main("/nonexistent/playwright-report.json")
assert rc == 0
assert "did not reach the e2e step" in out.getvalue()
if __name__ == "__main__":
for name, fn in sorted(globals().items()):
if name.startswith("test_") and callable(fn):
fn()
print(f" ok {name}")
print("playwright-summary self-check passed")
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""Self-check for the portals' Caddyfiles — stdlib asserts, no framework.
Run: python3 infra/test_portal_caddyfiles.py (also runs in `make unit`).
Each portal serves its Angular app and reverse-proxies *its own* BFF endpoint group
same-origin, so the browser never sees CORS and the DigiD token rides along (ADR-0010).
The four files are near-identical, which makes a copy-paste slip cheap to introduce and
expensive to find: proxying another portal's group hands a behandelaar's browser an
endpoint its token isn't for, and the failure shows up as a 401 three services away.
What is asserted per portal: it proxies exactly its own groups to the BFF service, and it
falls back to index.html so Angular's client-side routes survive a deep link / refresh.
"""
import os
import re
APPS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "apps")
# The self-service portal also renders the public register (S-09), so it proxies both.
EXPECTED = {
"self-service": {"/self-service/*", "/openbaar/*"},
"openbaar": {"/openbaar/*"},
"behandel": {"/behandel/*"},
"beheer": {"/beheer/*"},
}
ALL_GROUPS = {g for groups in EXPECTED.values() for g in groups}
def caddyfile(app):
with open(os.path.join(APPS, app, "Caddyfile")) as fh:
return fh.read()
def proxied_groups(text):
"""The path groups routed to the BFF: `handle <path> { reverse_proxy bff:8080 }`."""
return {
m.group(1)
for m in re.finditer(r"handle\s+(\S+)\s*\{[^}]*reverse_proxy\s+bff:8080", text)
}
def test_each_portal_proxies_exactly_its_own_endpoint_groups():
for app, expected in EXPECTED.items():
got = proxied_groups(caddyfile(app))
assert got == expected, f"{app}: proxies {got or '{}'}, expected {expected}"
def test_no_portal_proxies_another_portals_group():
for app, expected in EXPECTED.items():
strays = proxied_groups(caddyfile(app)) & (ALL_GROUPS - expected)
assert not strays, f"{app}: proxies another portal's group {strays}"
def test_every_portal_falls_back_to_index_html():
"""Angular routes client-side: an unknown path must serve the app, not a 404."""
for app in EXPECTED:
text = caddyfile(app)
assert "try_files {path} /index.html" in text, f"{app}: no SPA fallback"
assert "file_server" in text, f"{app}: nothing serves the built app"
if __name__ == "__main__":
for name, fn in sorted(globals().items()):
if name.startswith("test_") and callable(fn):
fn()
print(f" ok {name}")
print("portal Caddyfile self-check passed")
+15
View File
@@ -59,6 +59,20 @@ def services_in_trace(trace_id):
return names
def tempo_ingest_state():
"""#156: distinguish a broken trace chain from Tempo dropping spans. `ingester_clients` is 0
when the distributor has evicted its (single, in-process) ingester over a failed loopback
health check pushes fail and spans are lost, which looks identical to missing instrumentation
from here. Diagnostics only; never fails the check."""
try:
for line in _get(f"{TEMPO}/metrics").decode().splitlines():
if line.startswith("tempo_distributor_ingester_clients "):
return f"tempo {line.strip()} (0 = no ingester in the pool — evicted, so pushes\n are failing and spans are being dropped; see #156)"
except Exception as e:
return f"tempo /metrics unreadable: {e}"
return "tempo_distributor_ingester_clients not reported"
def main():
deadline = time.time() + TIMEOUT
generate_traffic()
@@ -74,6 +88,7 @@ def main():
generate_traffic()
print(f"FAIL — no single trace spanned {sorted(WANT)}; services seen: {sorted(seen)}",
file=sys.stderr)
print(f" {tempo_ingest_state()}", file=sys.stderr)
return 1
+7 -3
View File
@@ -15,9 +15,13 @@ set -euo pipefail
timeout="${WAIT_TIMEOUT:-420}"
deadline=$(( $(date +%s) + timeout ))
# compose service name -> container id. The name filter matches both docker
# compose ("infra-openzaak-1") and podman-compose ("infra_openzaak_1") naming.
cid_for() { docker ps -aq --filter "name=$1" | head -1; }
# compose service name -> container id. `--filter name=` is a substring match, so it is anchored on
# the compose replica suffix — otherwise 'objecten' also matches objecten-db / objecten-redis /
# objecten-celery, and 'objecttypen' matches objecttypen-db. Whichever docker listed first won, so a
# service with a sibling that has no healthcheck timed out with status=none while it was in fact
# healthy. The pattern matches both docker compose ("infra-objecten-1") and podman-compose
# ("infra_objecten_1") naming; the same anchoring the verify check scripts use.
cid_for() { docker ps -aq --filter "name=$1[-_][0-9]+\$" | head -1; }
for svc in "$@"; do
echo "waiting for '$svc' to be healthy (timeout ${timeout}s)..."
+1 -1
View File
@@ -15,7 +15,7 @@ export interface DigiadAuthOptions {
redirectUrl: string;
/**
* Route prefixes whose requests get the bearer token attached. The api-client calls the BFF with
* **relative** URLs (same-origin via the nginx proxy), so these must be relative path prefixes
* **relative** URLs (same-origin via the Caddy proxy), so these must be relative path prefixes
* (e.g. `/self-service/`) angular-auth-oidc-client matches `req.url.startsWith(route)`, and a
* relative `req.url` never starts with an absolute origin.
*/
@@ -10,7 +10,7 @@ export interface MedewerkerAuthOptions {
redirectUrl: string;
/**
* Route prefixes whose requests get the bearer token attached. The api-client calls the BFF with
* **relative** URLs (same-origin via the nginx proxy), so these must be relative path prefixes
* **relative** URLs (same-origin via the Caddy proxy), so these must be relative path prefixes
* (e.g. `/behandel/`) angular-auth-oidc-client matches `req.url.startsWith(route)`, and a
* relative `req.url` never starts with an absolute origin.
*/
+47 -1
View File
@@ -32,17 +32,63 @@ nav:
- "ADR-0008: Read projection store": architecture/adr-0008-read-projection-store.md
- "ADR-0009: External-task job worker": architecture/adr-0009-external-task-job-worker.md
- "ADR-0010: BFF OIDC validation": architecture/adr-0010-bff-oidc.md
- "ADR-0011: Approval status flow": architecture/adr-0011-approval-status-flow.md
- "ADR-0012: Citizen reference correlation": architecture/adr-0012-citizen-reference-correlation.md
- "ADR-0013: Behandel-portal wiring": architecture/adr-0013-behandel-portal-wiring.md
- "ADR-0014: Withdrawal cancels the process": architecture/adr-0014-withdrawal-cancels-the-process.md
- "ADR-0015: Beoordeling escalation": architecture/adr-0015-beoordeling-escalation.md
- "ADR-0016: Diploma eligibility DMN": architecture/adr-0016-diploma-eligibility-dmn.md
- "ADR-0017: Document-wait timeout": architecture/adr-0017-document-wait-timeout-cancellation.md
- "ADR-0018: Diploma upload via the ACL": architecture/adr-0018-diploma-upload-via-acl-documenten.md
- "ADR-0019: Zaak cancellation on timeout": architecture/adr-0019-zaak-cancellation-on-timeout.md
- "ADR-0020: Local stack self-seeds": architecture/adr-0020-local-stack-self-seeds.md
- "ADR-0021: Zaaktype by identificatie": architecture/adr-0021-acl-resolves-zaaktype-by-identificatie.md
- "ADR-0022: Quartz scheduler": architecture/adr-0022-quartz-scheduler.md
- "ADR-0023: Observability stack": architecture/adr-0023-observability-stack.md
- "ADR-0024: Prometheus AspNetCore exporter": architecture/adr-0024-prometheus-aspnetcore-exporter.md
- "ADR-0025: BFF reads catalogus via the ACL": architecture/adr-0025-bff-reads-catalogus-via-acl.md
- "ADR-0026: Mutable default-fill store": architecture/adr-0026-mutable-default-fill-store.md
- "ADR-0027: RegisterRecord objecttype": architecture/adr-0027-registerrecord-objecttype-schema.md
- "ADR-0028: Objecten holds the register": architecture/adr-0028-objecten-holds-the-register.md
- "ADR-0029: Objecten publishes to NRC": architecture/adr-0029-objecten-publishes-to-nrc.md
- "ADR-0030: Projection sourced from the register": architecture/adr-0030-projection-sourced-from-the-register.md
- "ADR-0031: MFA on the medewerker realm": architecture/adr-0031-mfa-on-the-medewerker-realm.md
- "ADR-0032: Werkbak live refresh": architecture/adr-0032-werkbak-live-refresh.md
- "ADR-0033: Kubernetes via one Helm chart": architecture/adr-0033-kubernetes-via-one-helm-chart.md
- "ADR-0034: Caddy serves the portals": architecture/adr-0034-caddy-serves-the-portals.md
- FDS-architectuur:
- Overzicht: architecture/fds/README.md
- Componentview (L3): architecture/fds/c4-component-view.md
- "Slice 1: walking skeleton": architecture/fds/slice-1-proposal.md
- "FDS ADR-0001: ACL op elke registergrens": architecture/fds/adr/0001-acl-at-every-register-boundary.md
- "FDS ADR-0002: FSC voor connectiviteit": architecture/fds/adr/0002-fsc-for-connectivity.md
- "FDS ADR-0003: PBAC via OPA": architecture/fds/adr/0003-pbac-via-opa.md
- "FDS ADR-0004: Begrensde cache": architecture/fds/adr/0004-bounded-cache.md
- "FDS ADR-0005: Verwerkingenlog via events": architecture/fds/adr/0005-ldv-verwerkingenlog.md
- "FDS ADR-0006: Modulegrens en hergebruik": architecture/fds/adr/0006-module-boundary-and-reuse.md
- "FDS ADR-template": architecture/fds/adr/template.md
- Working in Gitea: gitea-workflow.md
- Frontend decisions: frontend-decisions.md
- Demo script: demo-script.md
- Synthetic data: synthetic-data.md
- Runbooks:
- CI: runbooks/ci.md
- OpenZaak: runbooks/openzaak.md
- Open Notificaties (NRC): runbooks/opennotificaties.md
- Keycloak: runbooks/keycloak.md
- Flowable: runbooks/flowable.md
- Kubernetes on Talos: runbooks/kubernetes-talos.md
- Gitea Actions gotchas: runbooks/gitea-actions-gotchas.md
markdown_extensions:
- admonition
- toc:
permalink: true
- pymdownx.superfences
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
# Many docs referenced by PRD.md land in later slices; don't fail the build on them.
validation:
+18
View File
@@ -42,7 +42,12 @@ builder.Services.AddSingleton<IDefaultFillStore>(sp =>
return new InMemoryDefaultFillStore(
new DefaultFillSettings(d.Bronorganisatie, d.VerantwoordelijkeOrganisatie, d.Vertrouwelijkheidaanduiding));
});
builder.Services.AddSingleton(sp => sp.GetRequiredService<IConfiguration>()
.GetSection("Acl:Objecten").Get<ObjectenOptions>()
?? throw new InvalidOperationException("Missing configuration section 'Acl:Objecten'"));
builder.Services.AddHttpClient<IZaakGateway, OpenZaakGateway>();
// The Objecten hop that writes the register record on approval (S-19a, ADR-0028).
builder.Services.AddHttpClient<IRegisterRecordGateway, ObjectenGateway>();
// Singleton so the resolved zaaktype/informatieobjecttype URLs are cached across requests (S-27).
builder.Services.AddSingleton<IZaaktypeCatalog, CachedZaaktypeCatalog>();
builder.Services.AddScoped<AclService>();
@@ -85,6 +90,16 @@ app.MapPost("/zaken/reference", async (ZaakReferenceRequest body, AclService acl
return Results.Ok(new { reference });
});
// Read the register record an object in Objecten holds. The Event Subscriber projects a register
// write from the notification NRC delivers, which carries only the object URL, and may not talk to
// Objecten itself (§8.1, ADR-0028/ADR-0030). 404 when the object holds no record — the subscriber
// treats that as "nothing to project" rather than an error (§8.6).
app.MapPost("/register-records/read", async (RegisterRecordReadRequest body, AclService acl, CancellationToken ct) =>
{
var record = await acl.GetRegisterRecordAsync(new Uri(body.ObjectUrl), ct);
return record is null ? Results.NotFound() : Results.Ok(record);
});
// Store an uploaded diploma against a zaak (S-10b): the domain sends the file as base64; the ACL
// creates the ZGW enkelvoudiginformatieobject and relates it to the zaak (§8.1). Returns its URL.
app.MapPost("/documenten", async (StoreDocumentRequest body, AclService acl, CancellationToken ct) =>
@@ -126,6 +141,9 @@ public sealed record CancelZaakRequest(string ZaakUrl);
public sealed record ZaakReferenceRequest(string ZaakUrl);
/// <summary>The object whose register record the Event Subscriber wants read back (S-19b-2).</summary>
public sealed record RegisterRecordReadRequest(string ObjectUrl);
public sealed record StoreDocumentRequest(string ZaakUrl, string ContentBase64, string FileName, string ContentType);
public partial class Program;
+47 -4
View File
@@ -2,7 +2,12 @@ namespace Acl.Application;
/// <summary>The ACL's single operation: open a zaak from a domain payload,
/// default-filling the ZGW-mandatory fields (ADR-0003).</summary>
public sealed class AclService(IZaakGateway gateway, IDefaultFillStore fill, IZaaktypeCatalog catalog, IClock clock)
public sealed class AclService(
IZaakGateway gateway,
IRegisterRecordGateway register,
IDefaultFillStore fill,
IZaaktypeCatalog catalog,
IClock clock)
{
public async Task<Uri> OpenZaakAsync(DomainRegistration registration, CancellationToken ct = default)
{
@@ -19,20 +24,58 @@ public sealed class AclService(IZaakGateway gateway, IDefaultFillStore fill, IZa
clock.Today,
registration.Reference);
return await gateway.OpenZaakAsync(request, ct);
var zaakUrl = await gateway.OpenZaakAsync(request, ct);
// The register — not ZGW — is what the read projection is sourced from (ADR-0028/ADR-0030),
// so the record exists from submission, not only from approval. Same two-writes-converging
// posture as ApproveZaakAsync: the upsert is keyed on the zaak id, so a retried submit
// updates the record rather than adding a second one (§8.6).
await register.UpsertAsync(
new RegisterRecord(ZaakId(zaakUrl), RegisterRecordStatus.Ingediend, registration.Reference), ct);
return zaakUrl;
}
/// <summary>
/// Approve a zaak: set it to the eindstatus of the BIG zaaktype (resolved by identificatie, S-27).
/// The domain hands over only the zaak URL; the ACL owns which statustype means "approved" (§8.1).
/// Approve a zaak: set it to the eindstatus of the BIG zaaktype (resolved by identificatie, S-27),
/// then write the register record to Objecten (S-19a). The domain hands over only the zaak URL; the
/// ACL owns which statustype means "approved" and what the register record looks like (§8.1).
/// </summary>
/// <remarks>
/// OpenZaak holds the process, Objecten holds the register (ADR-0028), so approval is two writes
/// across two modules and is eventually consistent by construction. Both are idempotent — a status
/// is a log entry, the record upsert is keyed on the zaak id — so a caller that retries a failed
/// approval converges rather than duplicating.
/// </remarks>
public async Task ApproveZaakAsync(Uri zaakUrl, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(zaakUrl);
await gateway.SetZaakToEindstatusAsync(zaakUrl, await catalog.GetZaaktypeUrlAsync(ct), clock.Today, ct);
await register.UpsertAsync(
new RegisterRecord(
ZaakId(zaakUrl),
RegisterRecordStatus.Ingeschreven,
await gateway.GetZaakIdentificatieAsync(zaakUrl, ct)),
ct);
}
/// <summary>
/// The register record held by an object in Objecten, for the Event Subscriber (S-19b-2). The
/// subscriber gets only an object URL on the notification and may not read Objecten itself
/// (§8.1, ADR-0028), so the ACL reads it back.
/// </summary>
public Task<RegisterRecord?> GetRegisterRecordAsync(Uri objectUrl, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(objectUrl);
return register.GetAsync(objectUrl, ct);
}
/// <summary>The zaak's UUID — the key the register record and the read projection rows share.</summary>
private static string ZaakId(Uri zaakUrl) => zaakUrl.Segments[^1].TrimEnd('/');
/// <summary>
/// Cancel a zaak on document-timeout expiry (S-10c): set it to the BIG zaaktype's cancellation
/// statustype + resultaat. The domain hands over only the zaak URL; the ACL owns which
@@ -0,0 +1,38 @@
namespace Acl.Application;
/// <summary>
/// Port to the Objecten API, which holds the authoritative register record (S-19a, ADR-0028).
/// Implemented in Infrastructure — as with ZGW, the ACL is the only code that talks to the
/// upstream Common Ground module (§8.1).
/// </summary>
public interface IRegisterRecordGateway
{
/// <summary>
/// Write the register record for a registration, creating it if absent and updating it if it
/// already exists. Idempotent on <see cref="RegisterRecord.Id"/>: a replayed approval updates
/// the existing object instead of creating a second one (§8.6).
/// </summary>
Task UpsertAsync(RegisterRecord record, CancellationToken ct = default);
/// <summary>
/// The register record held by the object at <paramref name="objectUrl"/>, or <c>null</c> if that
/// object holds none. The Event Subscriber projects a register write from the notification NRC
/// delivers, which carries only the object URL — so it reads the record back through the ACL
/// rather than talking to Objecten itself (§8.1, S-19b-2).
/// </summary>
Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default);
}
/// <summary>
/// The public-safe register record, matching the <c>RegisterRecord</c> objecttype schema registered
/// in S-18c (ADR-0027). No bsn, no name — the register is world-readable.
/// </summary>
public sealed record RegisterRecord(string Id, string Status, string? Reference);
/// <summary>The register statuses the RegisterRecord objecttype's schema allows (ADR-0027).</summary>
public static class RegisterRecordStatus
{
public const string Ingediend = "INGEDIEND";
public const string Ingeschreven = "INGESCHREVEN";
}
@@ -0,0 +1,191 @@
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json.Serialization;
using Acl.Application;
namespace Acl.Infrastructure;
/// <summary>
/// The only code that talks to the Objecten API (ADR-0028). Writes the register record as an object
/// of the <c>RegisterRecord</c> objecttype registered in S-18c.
/// </summary>
public sealed class ObjectenGateway(HttpClient http, ObjectenOptions options, IClock clock) : IRegisterRecordGateway
{
// The objecttype URL + version are assigned by Objecttypen at seed time, so they are resolved by
// name on first use rather than pinned in config (same reasoning as ADR-0021).
// ponytail: memoised per instance only — the gateway is a transient typed client, so in practice
// that is one extra GET per approval against a neighbouring container. Lift it into a singleton
// cache (as CachedZaaktypeCatalog does for ZGW) if approvals ever get hot.
private Objecttype? objecttype;
public async Task UpsertAsync(RegisterRecord record, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(record);
var type = objecttype ??= await ResolveObjecttypeAsync(ct);
var existing = await FindExistingAsync(type.Url, record.Id, ct);
var data = new RecordDataDto(record.Id, record.Status, record.Reference);
// No existing object → create; otherwise PATCH, which appends a new record version to the same
// object. Either way the register ends up with exactly one object per registration (§8.6).
if (existing is null)
await SendAsync(HttpMethod.Post, new Uri(options.BaseUrl, "/api/v2/objects"),
new CreateObjectDto(type.Url.ToString(), NewRecord(type.Version, data)),
"Creating the register record", ct);
else
await SendAsync(HttpMethod.Patch, existing,
new PatchObjectDto(NewRecord(type.Version, data)),
"Updating the register record", ct);
}
public async Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default)
{
ArgumentNullException.ThrowIfNull(objectUrl);
// Fetched by the URL the notification carried, so no objecttype resolution and no search —
// unlike a write, which has to find the object for a registration id.
using var message = new HttpRequestMessage(HttpMethod.Get, objectUrl);
message.Headers.Authorization = new AuthenticationHeaderValue("Token", options.Token);
message.Headers.Add("Accept-Crs", "EPSG:4326");
using var response = await http.SendAsync(message, ct);
// The object may be gone by the time a (possibly redelivered) notification is handled —
// there is simply nothing to project, which is not a failure (§8.6).
if (response.StatusCode == HttpStatusCode.NotFound)
return null;
await EnsureSuccessAsync(response, "Reading the register record", ct);
var body = await response.Content.ReadFromJsonAsync<ReadObjectDto>(ct)
?? throw new InvalidOperationException("Objecten returned an empty object response");
var data = body.Record?.Data;
return data is null ? null : new RegisterRecord(data.Id, data.Status, data.Reference);
}
private RecordDto NewRecord(int typeVersion, RecordDataDto data) =>
new(typeVersion, data, clock.Today.ToString("yyyy-MM-dd"));
/// <summary>The URL + latest published version of the configured objecttype, read from Objecttypen.</summary>
private async Task<Objecttype> ResolveObjecttypeAsync(CancellationToken ct)
{
var page = await GetAsync<ObjecttypePage>(
new Uri(options.ObjecttypenBaseUrl, "/api/v2/objecttypes"),
options.ObjecttypenToken, crs: false, "objecttypen", ct);
var match = (page.Results ?? []).FirstOrDefault(o => o.Name == options.ObjecttypeName)
?? throw new InvalidOperationException(
$"No objecttype '{options.ObjecttypeName}' registered in Objecttypen — is the RegisterRecord seed applied?");
// Write against the highest *published* version: a draft version's schema is still being
// shaped, and objects written against it would be validated by a moving target. The objecttype
// carries its versions as URLs, so each is fetched for its status (the collection response
// gives no status) — once per gateway instance, alongside the lookup above.
var latest = 0;
foreach (var versionUrl in match.Versions ?? [])
{
var version = await GetAsync<ObjecttypeVersionDto>(
new Uri(versionUrl), options.ObjecttypenToken, crs: false, "objecttype version", ct);
if (version.Status == "published" && version.Version > latest)
latest = version.Version;
}
if (latest == 0)
throw new InvalidOperationException($"Objecttype '{options.ObjecttypeName}' has no published version");
return new Objecttype(new Uri(match.Url), latest);
}
/// <summary>The URL of the object already holding this registration's record, or null if there is none.</summary>
private async Task<Uri?> FindExistingAsync(Uri objecttypeUrl, string id, CancellationToken ct)
{
var query = new Uri(options.BaseUrl,
"/api/v2/objects?type=" + Uri.EscapeDataString(objecttypeUrl.ToString()) +
"&data_attrs=id__exact__" + Uri.EscapeDataString(id));
var page = await GetAsync<ObjectPage>(query, options.Token, crs: true, "objects", ct);
var match = (page.Results ?? []).FirstOrDefault();
return match is null ? null : new Uri(match.Url);
}
private async Task<T> GetAsync<T>(Uri uri, string token, bool crs, string label, CancellationToken ct)
{
using var message = new HttpRequestMessage(HttpMethod.Get, uri);
message.Headers.Authorization = new AuthenticationHeaderValue("Token", token);
if (crs)
message.Headers.Add("Accept-Crs", "EPSG:4326");
using var response = await http.SendAsync(message, ct);
await EnsureSuccessAsync(response, $"Querying {label}", ct);
return await response.Content.ReadFromJsonAsync<T>(ct)
?? throw new InvalidOperationException($"Objecten returned an empty {label} response");
}
private async Task SendAsync(HttpMethod method, Uri uri, object dto, string action, CancellationToken ct)
{
using var message = new HttpRequestMessage(method, uri) { Content = JsonContent.Create(dto) };
message.Headers.Authorization = new AuthenticationHeaderValue("Token", options.Token);
// The Objecten API is a geo API: it requires the CRS headers on reads and writes alike.
message.Headers.Add("Accept-Crs", "EPSG:4326");
message.Content.Headers.Add("Content-Crs", "EPSG:4326");
// As with OpenZaak, Objecten runs behind uwsgi, which rejects a chunked request body.
await message.Content.LoadIntoBufferAsync(ct);
using var response = await http.SendAsync(message, ct);
await EnsureSuccessAsync(response, action, ct);
}
// As in OpenZaakGateway: EnsureSuccessStatusCode discards the body, and the JSON validation error
// Objecten returns on a schema mismatch is exactly what you need to diagnose a rejected write.
private static async Task EnsureSuccessAsync(HttpResponseMessage response, string action, CancellationToken ct)
{
if (response.IsSuccessStatusCode)
return;
var body = await response.Content.ReadAsStringAsync(ct);
throw new HttpRequestException($"{action} failed: {(int)response.StatusCode} {response.ReasonPhrase}. {body}");
}
private sealed record Objecttype(Uri Url, int Version);
private sealed record ObjecttypePage(
[property: JsonPropertyName("results")] IReadOnlyList<ObjecttypeDto>? Results);
private sealed record ObjecttypeDto(
[property: JsonPropertyName("url")] string Url,
[property: JsonPropertyName("name")] string? Name,
[property: JsonPropertyName("versions")] IReadOnlyList<string>? Versions);
private sealed record ObjecttypeVersionDto(
[property: JsonPropertyName("version")] int Version,
[property: JsonPropertyName("status")] string? Status);
private sealed record ObjectPage(
[property: JsonPropertyName("results")] IReadOnlyList<ObjectDto>? Results);
private sealed record ObjectDto(
[property: JsonPropertyName("url")] string Url);
private sealed record ReadObjectDto(
[property: JsonPropertyName("record")] ReadRecordDto? Record);
private sealed record ReadRecordDto(
[property: JsonPropertyName("data")] RecordDataDto? Data);
private sealed record CreateObjectDto(
[property: JsonPropertyName("type")] string Type,
[property: JsonPropertyName("record")] RecordDto Record);
private sealed record PatchObjectDto(
[property: JsonPropertyName("record")] RecordDto Record);
private sealed record RecordDto(
[property: JsonPropertyName("typeVersion")] int TypeVersion,
[property: JsonPropertyName("data")] RecordDataDto Data,
[property: JsonPropertyName("startAt")] string StartAt);
private sealed record RecordDataDto(
[property: JsonPropertyName("id")] string Id,
[property: JsonPropertyName("status")] string Status,
[property: JsonPropertyName("reference")] string? Reference);
}
@@ -0,0 +1,20 @@
namespace Acl.Infrastructure;
/// <summary>
/// Connection + credential config for the Objecten and Objecttypen APIs. Both authenticate with a
/// static <c>Authorization: Token …</c> (they are not ZGW JWT APIs), so there is no client-id/secret
/// pair as with OpenZaak.
/// </summary>
public sealed class ObjectenOptions
{
public required Uri BaseUrl { get; init; }
public required string Token { get; init; }
/// <summary>Objecttypen API root — the ACL resolves the objecttype URL + version from it by name
/// rather than pinning a seed-time UUID in config (same reasoning as ADR-0021).</summary>
public required Uri ObjecttypenBaseUrl { get; init; }
public required string ObjecttypenToken { get; init; }
/// <summary>The objecttype the register record is written as (S-18c registers "RegisterRecord").</summary>
public required string ObjecttypeName { get; init; }
}
@@ -0,0 +1,105 @@
using Acl.Application;
using Acl.Infrastructure;
namespace Acl.IntegrationTests;
/// <summary>
/// S-19a (#149): the ObjectenGateway against a *real* Objecten + Objecttypen pair. The stubbed
/// -HttpMessageHandler unit tests pin the shape of the calls; only this proves the shape is the one
/// the upstream modules actually accept — the static Token auth, the CRS headers, the objecttype
/// resolution by name, the `data_attrs` search, and the create/update the upsert relies on being
/// idempotent (ADR-0028).
/// </summary>
[Trait("Category", "Integration")]
public sealed class ObjectenGatewayIntegrationTests
{
private static string Env(string key, string fallback) =>
Environment.GetEnvironmentVariable(key) is { Length: > 0 } v ? v : fallback;
private static ObjectenGateway Gateway() => new(
new HttpClient(),
new ObjectenOptions
{
BaseUrl = new(Env("OBJECTEN_BASE", "http://objecten.local:8000")),
Token = Env("OBJECTEN_TOKEN", "1234567890abcdef1234567890abcdef12345678"),
ObjecttypenBaseUrl = new(Env("OBJECTTYPEN_BASE", "http://objecttypen:8000")),
ObjecttypenToken = Env("OBJECTTYPEN_TOKEN", "0123456789abcdef0123456789abcdef01234567"),
ObjecttypeName = "RegisterRecord",
},
new SystemClock());
[Fact]
public async Task Writes_a_register_record_and_updates_it_in_place_on_a_second_write()
{
var gateway = Gateway();
// A key no other run shares: the verify stack is shared and keeps records between checks.
var id = Guid.NewGuid().ToString();
await gateway.UpsertAsync(new RegisterRecord(id, RegisterRecordStatus.Ingediend, "INT-TEST-1"));
await gateway.UpsertAsync(new RegisterRecord(id, RegisterRecordStatus.Ingeschreven, "INT-TEST-1"));
var records = await ReadAllAsync(id);
var only = Assert.Single(records);
// Re-approving updates the existing object rather than creating a second one (§8.6).
Assert.Equal(RegisterRecordStatus.Ingeschreven, only.Status);
Assert.Equal("INT-TEST-1", only.Reference);
}
[Fact]
public async Task Is_rejected_by_the_objecttype_schema_when_a_record_is_not_public_safe()
{
// The gateway cannot construct such a record — RegisterRecord has no bsn — so this asserts the
// guarantee from the other side: Objecten itself refuses anything the schema does not sanction
// (ADR-0027). Posted raw, exactly as the gateway would post a record.
var gateway = Gateway();
var id = Guid.NewGuid().ToString();
await gateway.UpsertAsync(new RegisterRecord(id, RegisterRecordStatus.Ingeschreven, "INT-TEST-2"));
var stored = Assert.Single(await ReadAllAsync(id));
Assert.Null(stored.Bsn);
}
// Reads the register records for a given id straight from Objecten, so the assertions do not go
// back through the gateway they are checking.
private static async Task<IReadOnlyList<StoredRecord>> ReadAllAsync(string id)
{
using var http = new HttpClient();
var objecttype = await ResolveObjecttypeUrlAsync(http);
var query = new Uri(new Uri(Env("OBJECTEN_BASE", "http://objecten.local:8000")),
"/api/v2/objects?type=" + Uri.EscapeDataString(objecttype) +
"&data_attrs=id__exact__" + Uri.EscapeDataString(id));
using var message = new HttpRequestMessage(HttpMethod.Get, query);
message.Headers.Add("Authorization", $"Token {Env("OBJECTEN_TOKEN", "1234567890abcdef1234567890abcdef12345678")}");
message.Headers.Add("Accept-Crs", "EPSG:4326");
using var response = await http.SendAsync(message);
response.EnsureSuccessStatusCode();
using var document = System.Text.Json.JsonDocument.Parse(await response.Content.ReadAsStringAsync());
return document.RootElement.GetProperty("results").EnumerateArray()
.Select(o => o.GetProperty("record").GetProperty("data"))
.Select(d => new StoredRecord(
d.GetProperty("status").GetString()!,
d.GetProperty("reference").GetString(),
d.TryGetProperty("bsn", out var bsn) ? bsn.GetString() : null))
.ToList();
}
private static async Task<string> ResolveObjecttypeUrlAsync(HttpClient http)
{
var query = new Uri(new Uri(Env("OBJECTTYPEN_BASE", "http://objecttypen:8000")), "/api/v2/objecttypes");
using var message = new HttpRequestMessage(HttpMethod.Get, query);
message.Headers.Add("Authorization", $"Token {Env("OBJECTTYPEN_TOKEN", "0123456789abcdef0123456789abcdef01234567")}");
using var response = await http.SendAsync(message);
response.EnsureSuccessStatusCode();
using var document = System.Text.Json.JsonDocument.Parse(await response.Content.ReadAsStringAsync());
return document.RootElement.GetProperty("results").EnumerateArray()
.First(o => o.GetProperty("name").GetString() == "RegisterRecord")
.GetProperty("url").GetString()!;
}
private sealed record StoredRecord(string Status, string? Reference, string? Bsn);
}
+104 -2
View File
@@ -75,6 +75,27 @@ public class AclServiceTests
Task.FromResult(Zaaktypen);
}
private sealed class FakeRegisterRecordGateway : IRegisterRecordGateway
{
public readonly List<RegisterRecord> Upserted = [];
public RegisterRecord? Stored;
public Uri? ReadFrom;
public Task UpsertAsync(RegisterRecord record, CancellationToken ct = default)
{
Upserted.Add(record);
return Task.CompletedTask;
}
public Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default)
{
ReadFrom = objectUrl;
return Task.FromResult(Stored);
}
}
private static AclDefaults Defaults() => new()
{
Bronorganisatie = "517439943",
@@ -88,7 +109,10 @@ public class AclServiceTests
new(new DefaultFillSettings(d.Bronorganisatie, d.VerantwoordelijkeOrganisatie, d.Vertrouwelijkheidaanduiding));
private static AclService ServiceWith(FakeGateway gateway, AclDefaults defaults, DateOnly today) =>
new(gateway, FillFrom(defaults), new CachedZaaktypeCatalog(gateway, defaults), new FixedClock(today));
ServiceWith(gateway, new FakeRegisterRecordGateway(), defaults, today);
private static AclService ServiceWith(FakeGateway gateway, FakeRegisterRecordGateway register, AclDefaults defaults, DateOnly today) =>
new(gateway, register, FillFrom(defaults), new CachedZaaktypeCatalog(gateway, defaults), new FixedClock(today));
private sealed class FixedClock(DateOnly today) : IClock
{
@@ -116,6 +140,52 @@ public class AclServiceTests
Assert.Equal("reg-77", req.Identificatie);
}
[Fact]
public async Task Opening_a_zaak_also_writes_an_ingediend_register_record(/* S-19b-2 */)
{
var gateway = new FakeGateway();
var register = new FakeRegisterRecordGateway();
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
await service.OpenZaakAsync(new DomainRegistration("123456782", "reg-77"));
// The register — not ZGW — is what the read projection is sourced from (ADR-0028), so a
// submitted registration has to exist there the moment the zaak is opened, not only on
// approval. Approval upserts this same record to INGESCHREVEN.
var record = Assert.Single(register.Upserted);
Assert.Equal("abc", record.Id);
Assert.Equal("INGEDIEND", record.Status);
// The reference comes from the registration itself — no ZGW read-back needed on this path.
Assert.Equal("reg-77", record.Reference);
}
[Fact]
public async Task Reading_a_register_record_goes_through_the_objecten_gateway(/* S-19b-2 */)
{
var gateway = new FakeGateway();
var register = new FakeRegisterRecordGateway { Stored = new RegisterRecord("abc", "INGESCHREVEN", "reg-77") };
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
var objectUrl = new Uri("http://objecten.local:8000/api/v2/objects/9de4a2ca");
var record = await service.GetRegisterRecordAsync(objectUrl);
Assert.Equal(objectUrl, register.ReadFrom);
Assert.Equal("abc", record!.Id);
Assert.Equal("INGESCHREVEN", record.Status);
Assert.Equal("reg-77", record.Reference);
}
[Fact]
public async Task Reading_a_register_record_from_a_null_url_is_rejected(/* S-19b-2 */)
{
var gateway = new FakeGateway();
var register = new FakeRegisterRecordGateway();
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
await Assert.ThrowsAsync<ArgumentNullException>(() => service.GetRegisterRecordAsync(null!));
Assert.Null(register.ReadFrom);
}
[Fact]
public async Task Opening_a_zaak_reflects_a_default_fill_update(/* S-15b */)
{
@@ -161,10 +231,42 @@ public class AclServiceTests
public async Task Approving_a_null_zaak_is_rejected_without_touching_the_gateway()
{
var gateway = new FakeGateway();
var service = ServiceWith(gateway, Defaults(), new DateOnly(2026, 6, 4));
var register = new FakeRegisterRecordGateway();
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
await Assert.ThrowsAsync<ArgumentNullException>(() => service.ApproveZaakAsync(null!));
Assert.Null(gateway.Approved);
Assert.Empty(register.Upserted);
}
[Fact]
public async Task Approving_a_zaak_writes_the_register_record_to_objecten(/* S-19a */)
{
var gateway = new FakeGateway();
var register = new FakeRegisterRecordGateway();
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
await service.ApproveZaakAsync(new Uri("http://openzaak/zaken/api/v1/zaken/abc"));
var record = Assert.Single(register.Upserted);
// The record is keyed on the zaak id — the same key the read projection rows carry (S-19b).
Assert.Equal("abc", record.Id);
Assert.Equal("INGESCHREVEN", record.Status);
// The public-safe reference comes from the zaak's identificatie, never from the domain payload.
Assert.Equal("REG-FROM-ZAAK", record.Reference);
}
[Fact]
public async Task Cancelling_a_zaak_writes_no_register_record(/* S-19a */)
{
var gateway = new FakeGateway();
var register = new FakeRegisterRecordGateway();
var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
await service.CancelZaakAsync(new Uri("http://openzaak/zaken/api/v1/zaken/abc"));
// Only an approval enters the register; a cancelled zaak never becomes a register record.
Assert.Empty(register.Upserted);
}
[Fact]
@@ -0,0 +1,367 @@
using System.Net;
using System.Net.Http.Json;
using Acl.Application;
using Acl.Infrastructure;
namespace Acl.Tests;
public class ObjectenGatewayTests
{
private sealed class StubHandler(Func<HttpRequestMessage, Task<HttpResponseMessage>> onSend)
: HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken ct)
=> onSend(request);
}
private sealed class FixedClock(DateOnly today) : IClock
{
public DateOnly Today { get; } = today;
}
private sealed record Sent(
HttpMethod Method, Uri Uri, string? Body, string? Auth, string? ContentCrs, string? AcceptCrs, long? ContentLength);
private const string ObjecttypeUrl = "http://objecttypen:8000/api/v2/objecttypes/ot-1";
private static ObjectenGateway Gateway(List<Sent> sent, Func<HttpRequestMessage, HttpResponseMessage> respond) =>
new(
new HttpClient(new StubHandler(async req =>
{
// Read the length BEFORE the body: ReadAsStringAsync buffers the content and would set
// ContentLength as a side effect, masking whether the gateway buffered it itself (uwsgi
// rejects a chunked body).
sent.Add(new Sent(
req.Method,
req.RequestUri!,
ContentLength: req.Content?.Headers.ContentLength,
Body: req.Content is null ? null : await req.Content.ReadAsStringAsync(),
Auth: req.Headers.Authorization?.ToString(),
ContentCrs: req.Content?.Headers.TryGetValues("Content-Crs", out var c) == true ? string.Join(",", c!) : null,
AcceptCrs: req.Headers.TryGetValues("Accept-Crs", out var a) ? string.Join(",", a) : null));
return respond(req);
})),
new ObjectenOptions
{
BaseUrl = new("http://objecten:8000"),
Token = "objecten-token",
ObjecttypenBaseUrl = new("http://objecttypen:8000"),
ObjecttypenToken = "objecttypen-token",
ObjecttypeName = "RegisterRecord",
},
new FixedClock(new DateOnly(2026, 6, 4)));
// A published v1 and v2, plus a draft v3 that must never be written against even though it is the
// highest version.
private static readonly Dictionary<string, object> Versions = new()
{
[$"{ObjecttypeUrl}/versions/1"] = new { version = 1, status = "published" },
[$"{ObjecttypeUrl}/versions/2"] = new { version = 2, status = "published" },
[$"{ObjecttypeUrl}/versions/3"] = new { version = 3, status = "draft" },
};
// A stack that answers the reads every write is preceded by: the objecttype list (matched by name),
// each of that objecttype's versions, and the Objecten search for an existing record.
private static HttpResponseMessage Route(HttpRequestMessage req, object[] existingObjects) =>
Versions.TryGetValue(req.RequestUri!.ToString(), out var version)
? Json(version)
: req.RequestUri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)
? Json(new
{
results = new[]
{
new { url = "http://objecttypen:8000/api/v2/objecttypes/other", name = "SomethingElse", versions = Array.Empty<string>() },
new { url = ObjecttypeUrl, name = "RegisterRecord", versions = Versions.Keys.ToArray() },
},
})
: req.Method == HttpMethod.Get
? Json(new { results = existingObjects })
: new HttpResponseMessage(HttpStatusCode.Created) { Content = JsonContent.Create(new { url = "http://objecten:8000/api/v2/objects/obj-1" }) };
private static HttpResponseMessage Json(object body) =>
new(HttpStatusCode.OK) { Content = JsonContent.Create(body) };
private static RegisterRecord Record() => new("zaak-uuid-1", RegisterRecordStatus.Ingeschreven, "REG-2026-0001");
[Fact]
public async Task Reads_a_register_record_back_from_its_object_url(/* S-19b-2 */)
{
var sent = new List<Sent>();
var objectUrl = new Uri("http://objecten:8000/api/v2/objects/obj-9");
var gateway = Gateway(sent, _ => Json(new
{
url = objectUrl.ToString(),
record = new { data = new { id = "zaak-uuid-1", status = "INGESCHREVEN", reference = "REG-2026-0001" } },
}));
var record = await gateway.GetAsync(objectUrl);
// The object is fetched directly by the URL the notification carried — no objecttype
// resolution and no search, unlike a write.
var read = Assert.Single(sent);
Assert.Equal(HttpMethod.Get, read.Method);
Assert.Equal(objectUrl, read.Uri);
// Objecten is a geo API: the CRS header is required on reads too.
Assert.Equal("EPSG:4326", read.AcceptCrs);
Assert.Equal("Token objecten-token", read.Auth);
Assert.Equal("zaak-uuid-1", record!.Id);
Assert.Equal("INGESCHREVEN", record.Status);
Assert.Equal("REG-2026-0001", record.Reference);
}
[Fact]
public async Task Reading_an_object_that_is_gone_yields_no_record(/* S-19b-2 */)
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.NotFound));
// A record deleted between the notification and the read is not an error — there is simply
// nothing to project (§8.6: the subscriber tolerates whatever order deliveries arrive in).
Assert.Null(await gateway.GetAsync(new Uri("http://objecten:8000/api/v2/objects/gone")));
}
[Fact]
public async Task Creates_the_object_when_none_exists_for_the_registration()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var write = sent.Single(s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.Contains($"\"type\":\"{ObjecttypeUrl}\"", write.Body);
// The highest *published* version (2), not the highest version (a draft 3).
Assert.Contains("\"typeVersion\":2", write.Body);
Assert.Contains("\"id\":\"zaak-uuid-1\"", write.Body);
Assert.Contains("\"status\":\"INGESCHREVEN\"", write.Body);
Assert.Contains("\"reference\":\"REG-2026-0001\"", write.Body);
Assert.Contains("\"startAt\":\"2026-06-04\"", write.Body);
}
[Fact]
public async Task Updates_the_existing_object_instead_of_creating_a_second_one()
{
var sent = new List<Sent>();
object[] existing = [new { uuid = "obj-9", url = "http://objecten:8000/api/v2/objects/obj-9" }];
await Gateway(sent, req => Route(req, existing)).UpsertAsync(Record());
Assert.DoesNotContain(sent, s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
var write = sent.Single(s => s.Method == HttpMethod.Patch);
Assert.Equal("http://objecten:8000/api/v2/objects/obj-9", write.Uri.ToString());
Assert.Contains("\"status\":\"INGESCHREVEN\"", write.Body);
}
[Fact]
public async Task Searches_objecten_for_the_registration_id_within_the_objecttype()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var search = sent.Single(s => s.Method == HttpMethod.Get && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.Contains("type=" + Uri.EscapeDataString(ObjecttypeUrl), search.Uri.Query);
Assert.Contains("data_attrs=id__exact__zaak-uuid-1", search.Uri.Query);
}
[Fact]
public async Task Authenticates_with_the_static_token_of_each_api()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)),
s => Assert.Equal("Token objecttypen-token", s.Auth));
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objects", StringComparison.Ordinal)),
s => Assert.Equal("Token objecten-token", s.Auth));
}
[Fact]
public async Task Sends_the_geo_crs_headers_the_objecten_api_requires()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var objects = sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objects", StringComparison.Ordinal)).ToList();
Assert.All(objects, s => Assert.Equal("EPSG:4326", s.AcceptCrs));
Assert.All(objects.Where(s => s.Body is not null), s => Assert.Equal("EPSG:4326", s.ContentCrs));
}
[Fact]
public async Task Resolves_the_objecttype_once_and_reuses_it_across_writes()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => Route(req, []));
await gateway.UpsertAsync(Record());
await gateway.UpsertAsync(Record() with { Id = "zaak-uuid-2" });
Assert.Single(sent, s => s.Uri.AbsolutePath == "/api/v2/objecttypes");
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_has_no_published_version()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.RequestUri!.AbsolutePath.Contains("/versions/", StringComparison.Ordinal)
? Json(new { version = 1, status = "draft" })
: Route(req, []));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("published version", error.Message);
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_is_not_registered()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = JsonContent.Create(new { results = Array.Empty<object>() }),
});
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("RegisterRecord", error.Message);
}
[Fact]
public async Task Surfaces_the_objecten_error_body_when_a_write_is_rejected()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Post && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? new HttpResponseMessage(HttpStatusCode.BadRequest) { Content = new StringContent("{\"detail\":\"schema mismatch\"}") }
: Route(req, []));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("schema mismatch", error.Message);
Assert.Contains("Creating the register record", error.Message);
}
[Fact]
public async Task Surfaces_the_objecten_error_body_when_an_update_is_rejected()
{
var sent = new List<Sent>();
object[] existing = [new { url = "http://objecten:8000/api/v2/objects/obj-9" }];
var gateway = Gateway(sent, req => req.Method == HttpMethod.Patch
? new HttpResponseMessage(HttpStatusCode.BadRequest) { Content = new StringContent("{\"detail\":\"stale version\"}") }
: Route(req, existing));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("stale version", error.Message);
Assert.Contains("Updating the register record", error.Message);
}
[Fact]
public async Task Surfaces_a_failed_read_instead_of_writing_blind()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.Unauthorized)
{
Content = new StringContent("{\"detail\":\"invalid token\"}"),
});
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("Querying objecttypen", error.Message);
Assert.Contains("invalid token", error.Message);
// A read that failed must never be mistaken for "nothing there yet" and followed by a write.
Assert.DoesNotContain(sent, s => s.Method == HttpMethod.Post || s.Method == HttpMethod.Patch);
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_carries_no_versions_at_all()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.RequestUri!.AbsolutePath == "/api/v2/objecttypes"
? Json(new { results = new[] { new { url = ObjecttypeUrl, name = "RegisterRecord" } } })
: Route(req, []));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("published version", error.Message);
}
[Fact]
public async Task Says_which_read_failed_when_the_objecten_search_errors()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Get && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? new HttpResponseMessage(HttpStatusCode.InternalServerError) { Content = new StringContent("boom") }
: Route(req, []));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("Querying objects", error.Message);
}
[Fact]
public async Task Surfaces_an_empty_read_body_rather_than_dereferencing_it()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("null", System.Text.Encoding.UTF8, "application/json"),
});
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("objecttypen", error.Message);
}
[Fact]
public async Task Treats_a_result_less_response_as_no_match_rather_than_crashing()
{
var sent = new List<Sent>();
// The objecttypes collection carries no `results` key — the objecttype is absent, which must
// surface as the "not registered" error rather than an ArgumentNullException from LINQ.
var gateway = Gateway(sent, _ => Json(new { }));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("RegisterRecord", error.Message);
}
[Fact]
public async Task Creates_the_object_when_the_search_response_carries_no_results_key()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Get && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? Json(new { })
: Route(req, []));
await gateway.UpsertAsync(Record());
Assert.Contains(sent, s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
}
[Fact]
public async Task Reads_objecttypen_without_the_crs_headers_it_does_not_accept()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
// Objecttypen is not a geo API; only the Objecten hops carry CRS.
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)),
s => Assert.Null(s.AcceptCrs));
}
[Fact]
public async Task Buffers_the_write_body_so_uwsgi_gets_a_content_length()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var write = sent.Single(s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.NotNull(write.ContentLength);
}
[Fact]
public async Task Rejects_a_null_record_without_calling_objecten()
{
var sent = new List<Sent>();
await Assert.ThrowsAsync<ArgumentNullException>(() => Gateway(sent, req => Route(req, [])).UpsertAsync(null!));
Assert.Empty(sent);
}
}

Some files were not shown because too many files have changed in this diff Show More