## What & why Two changes, made and verified together on a real cluster. **S-24 / #25 — a Helm chart for the platform.** One chart, `infra/helm/big-reference`, whose `values.yaml` is a near-literal transcription of `infra/docker-compose.yml`, rendered by three generic templates (Deployment, Job, Service) over a `workloads` map. Adding a service is a values edit. `make k8s-lint` renders and schema-checks the whole stack without a cluster. The issue asked for a *sketch*; this is deployed and verified end to end (see below), which is more than it asked for — the part it asked for that is **not** here is the production-posture write-up (HA, secrets, backup), see Known gaps. **#166 — Caddy replaces nginx in the portals.** nginx resolves a variable `proxy_pass` upstream itself, using only the `resolver` directive and never `/etc/resolv.conf`'s search domains. That had cost two workarounds in one script: rewriting the resolver address for rootless podman, and injecting a full FQDN so the bare `bff` name could resolve on Kubernetes. Caddy dials per request through the system resolver, so `reverse_proxy bff:8080` works on every engine unchanged; `apps/portal-nginx-resolver.sh` and the chart's `BFF_HOST` env are deleted. Closes #25 Closes #166 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation — twice: the Caddyfile contract test before the Caddyfiles, `make k8s-lint` before the chart. - [x] Implementation makes the test pass. - [x] Conventional Commits referencing the issues (`refs #25` / `refs #166`). - [ ] CI green — awaiting the run on this PR (`make k8s-lint`, `dotnet format` and the new unit self-check pass locally; the compose e2e and mutation lanes are CI's). - [ ] `docker compose up` from a fresh clone reaches green health checks within 3 minutes — the portal images were rebuilt and verified standalone, but a full `make up` run has not been done on this branch. Please confirm in review or let CI's smoke test speak. - [x] Docs updated — `docs/runbooks/kubernetes-talos.md` (new), `frontend-decisions.md`, `demo-script.md`, and the docs that named nginx. - [x] ADR added — ADR-0033 (chart) and ADR-0034 (Caddy). - [ ] Demo note in `docs/demo-script.md` — not added: the deployment target is not a user-visible slice, and the Caddy swap is invisible to the demo script beyond the wording fix included here. ## How it was verified Brought up from scratch on a single-node Talos v1.14.0 VM (6 vCPU / 10 GB, virtio disk) under virt-manager: **29 pods ready and four bootstrap Jobs complete in under three minutes, zero restarts**, using ~4.4 GB of the VM's 10 GB. - Full Common Ground path: portal Caddy → BFF → domain → Flowable → ACL → OpenZaak + Objecten → NRC → event-subscriber → projection → public register (`INGEDIEND`, reference matching the submitted registration). - Werkbak read with an MFA'd medewerker token → 200. - The browser flow driven with Playwright against `http://localhost:30140`: secure context, `crypto.subtle` present, Keycloak form reached, login completed, **no console errors**. - Routing checked against a stub BFF: SPA fallback serves deep links, each portal proxies its own groups, and a portal does *not* proxy a neighbour's group. ## Notes for reviewers Three bugs this shook out, each fixed at the cause rather than the symptom: 1. **`command` vs `args`.** Compose's `command:` replaces the image CMD; Kubernetes' replaces the ENTRYPOINT. Transcribing one to the other broke every upstream image that relies on its entrypoint — postgres refused to run as root, Keycloak tried to exec `start-dev`. The chart now `fail`s at render time on `command`. 2. **Concurrent migrations.** Both `/setup_configuration.sh` and `/start.sh` run `manage.py migrate`; compose serialises them with `depends_on`, Kubernetes has no such edge, so the init Job and its web pod raced (`relation "zgw_consumers_service" already exists`). The four Django services now do both steps in order in the web pod — which also deletes four workloads. 3. **`emptyDir` databases are wiped by any pod-template change.** `make k8s-reseed` now also restarts `event-subscriber` and `projection-api`, which create the projection schema on start and otherwise keep writing to a schema-less database. Known gaps / follow-ups: - **Secrets.** `values.yaml` carries the dev credentials in plain text (`admin/admin`, the ZGW client secret, the two Objecten tokens) and the chart has no `Secret` objects. Fine for a laptop demo, and exactly what #25's "production posture" ADR should address — I suggest a follow-up issue rather than stretching this PR. - **No CI gate for the chart yet.** `make k8s-lint` exists but is not wired into `.gitea/workflows/ci.yaml`, and nothing enforces that the chart and the compose file stay in step. Worth a small follow-up. - **This is two slices in one PR.** They were built and verified together and the diff is entangled (the chart was written against Caddy from the start), so splitting now would mean re-creating an nginx-shaped chart to throw away. Happy to split if you'd rather. - **Rebased onto #161** (merged as #165) rather than merged, to keep the history linear. One conflict, in the `unit:` target where both branches add a self-check line — resolved by keeping both. #161's `infra/host-browser.yml` arrived with `/usr/share/nginx/html/config.json` and is fixed to `/usr/share/caddy/` inside the `feat(portals)` commit, so no commit on this branch leaves that overlay pointing at a path the images no longer have.Reviewed-on: #167
170 lines
9.6 KiB
TypeScript
170 lines
9.6 KiB
TypeScript
import { expect, request, test } from '@playwright/test';
|
|
import { loginBurger, loginMedewerker } from './keycloak-login';
|
|
|
|
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
|
|
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
|
|
// appears in the openbaar register as INGEDIEND; the citizen supplies the documents the process is
|
|
// waiting for (S-10a); a behandelaar then logs in to the behandel portal, finds the registration in
|
|
// the werkbak, and approves it (goedkeuren); the decision completes the Flowable Beoordelen task and
|
|
// flows via the ACL → Objecten → NRC → event-subscriber → projection, and the openbaar register
|
|
// shows INGESCHREVEN.
|
|
//
|
|
// Since ADR-0030 both public statuses come from the register in Objecten, not from ZGW zaak events:
|
|
// the ACL writes the record on submit (INGEDIEND) and upserts it on approval (INGESCHREVEN), so the
|
|
// INGEDIEND assertion below is itself proof of the re-sourced path.
|
|
test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt → public INGESCHREVEN', async ({
|
|
page,
|
|
context,
|
|
}) => {
|
|
// Visiting the guarded page redirects to the Keycloak (mock DigiD) login.
|
|
await page.goto('/');
|
|
|
|
// Keycloak's default login form (stable ids across themes). Its own DigiD user: the verify-* API
|
|
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
|
|
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
|
|
// uses a dedicated citizen no other actor touches.
|
|
await loginBurger(page, 'emma-burger');
|
|
|
|
// Back on the portal, authenticated.
|
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
|
|
|
await page.getByRole('button', { name: /indienen/i }).click();
|
|
|
|
// The BFF accepted it and the page shows the confirmation with the reference.
|
|
const confirmation = page.getByText(/ontvangen/i);
|
|
await expect(confirmation).toBeVisible();
|
|
const reference = (await confirmation.textContent())?.match(/Referentie:\s*([0-9a-fA-F-]+)/)?.[1];
|
|
expect(reference, 'the confirmation shows a registration reference').toBeTruthy();
|
|
|
|
// The openbaar register (anonymous, its own origin) shows the submitted entry once the projection
|
|
// catches up. We check it on a SEPARATE page so the self-service tab keeps its (in-memory) submitted
|
|
// state — the "Documenten aanleveren" action below acts on that same session. The projection updates
|
|
// asynchronously (NRC → event-subscriber), so reload until *this* submission's row appears. We poll
|
|
// on the reference cell (not a generic INGEDIEND cell): the shared verify stack already holds
|
|
// INGEDIEND rows from earlier checks, so a status-only poll would short-circuit on a stale row.
|
|
const staff = await context.newPage();
|
|
await staff.goto('http://openbaar/');
|
|
await expect(staff.getByRole('heading', { name: /Openbaar BIG-register/i })).toBeVisible();
|
|
|
|
// #78: the reference shown in the public register must be the exact one the citizen saw on the
|
|
// submit confirmation — no mismatch between the two portals.
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('cell', { name: reference }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
await expect(staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGEDIEND' }))
|
|
.toBeVisible();
|
|
|
|
// A behandelaar opens the behandel-portal werkbak and approves the registration (goedkeuren) — the
|
|
// S-12 flow that replaces the temporary admin endpoint. The staff tab switches to the medewerker
|
|
// realm (a different Keycloak realm than the citizen's digid session).
|
|
//
|
|
// The werkbak is opened BEFORE the citizen supplies the documents that route the registration to
|
|
// Beoordelen, so its row cannot be there at page load: the only thing that can deliver it to this
|
|
// already-open page is the werkbak refreshing itself (S-26/#162, ADR-0032). This spec used to
|
|
// `staff.reload()` in a poll loop here; the absence of that reload is the live-refresh assertion.
|
|
await staff.goto('http://behandel/');
|
|
// That realm enforces MFA (S-15c), so the behandelaar logs in with password + TOTP.
|
|
await loginMedewerker(staff, 'merel-behandelaar');
|
|
|
|
await expect(staff.getByRole('heading', { name: /Werkbak/i })).toBeVisible();
|
|
|
|
// Target the decide button by reference (not a generic "Goedkeuren"): the shared verify stack holds
|
|
// other open tasks, so a positional match could act on someone else's registration.
|
|
const goedkeuren = staff.getByRole('button', { name: `Goedkeuren ${reference}` });
|
|
await expect(goedkeuren, 'the registration is not awaiting beoordeling yet').toBeHidden();
|
|
|
|
// Provide the documents the registration is waiting for (S-10a), on the still-open self-service tab.
|
|
// The process parks at WachtOpDocumenten only after the zaak is opened; the INGEDIEND row above proves
|
|
// the zaak exists — so the OpenZaak worker has completed and the process is now at the wait — which is
|
|
// why we supply the documents here rather than right after submit, when the trigger would race the
|
|
// wait and no-op. (S-10b turns this into a real file upload; here it is the trigger that unblocks
|
|
// beoordeling.)
|
|
await page.bringToFront();
|
|
await page.setInputFiles('#diploma', {
|
|
name: 'diploma.pdf',
|
|
mimeType: 'application/pdf',
|
|
buffer: Buffer.from('%PDF-1.4 synthetic diploma\n'),
|
|
});
|
|
await page.getByRole('button', { name: /documenten aanleveren/i }).click();
|
|
await expect(page.getByText(/documenten zijn aangeleverd/i)).toBeVisible();
|
|
|
|
// Back to the werkbak — untouched since login, never reloaded. The row arrives on its own once the
|
|
// DMN routes the registration to Beoordelen. (Foregrounded so Chromium doesn't throttle the page's
|
|
// refresh timer as a hidden tab.)
|
|
await staff.bringToFront();
|
|
await expect(goedkeuren).toBeVisible({ timeout: 30_000 });
|
|
|
|
// Click and wait for the decide POST to finish (204) BEFORE leaving the page. `click()` only
|
|
// dispatches the request; navigating away immediately cancels it in flight (the proxy logs a client-cancelled request) and
|
|
// the decision never reaches the domain — so the registration would stay INGEDIEND.
|
|
const decided = staff.waitForResponse(
|
|
(r) =>
|
|
r.url().includes(`/behandel/registrations/${reference}/decide`) &&
|
|
r.request().method() === 'POST',
|
|
);
|
|
await goedkeuren.click();
|
|
expect((await decided).status()).toBe(204);
|
|
|
|
// The approval flows back to the projection; back on the openbaar register *our* row (matched by
|
|
// its reference) now shows INGESCHREVEN.
|
|
await staff.goto('http://openbaar/');
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGESCHREVEN' }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
|
|
// S-19a: the same approval also wrote the canonical register record to Objecten (ADR-0028).
|
|
// Asserted here rather than in verify-domain because this is the only check that drives a *real*
|
|
// approval — verify-domain completes the Beoordelen task straight through Flowable REST, which
|
|
// bypasses the domain `decide` path that calls the ACL.
|
|
const records = await registerRecordsFor(reference);
|
|
// Matched on OUR reference: the verify stack is shared and holds records from earlier checks.
|
|
expect(records, `expected exactly one RegisterRecord for ${reference}`).toHaveLength(1);
|
|
expect(records[0].status).toBe('INGESCHREVEN');
|
|
// The register is world-readable, so the record must carry nothing but the public-safe fields
|
|
// (ADR-0027) — Objecten's own schema validation enforces this, and this proves it end to end.
|
|
expect(Object.keys(records[0]).sort()).toEqual(['id', 'reference', 'status']);
|
|
});
|
|
|
|
const OBJECTEN = process.env.OBJECTEN_URL ?? 'http://objecten:8000';
|
|
const OBJECTTYPEN = process.env.OBJECTTYPEN_URL ?? 'http://objecttypen:8000';
|
|
const OBJECTEN_TOKEN = process.env.OBJECTEN_TOKEN ?? '1234567890abcdef1234567890abcdef12345678';
|
|
const OBJECTTYPEN_TOKEN = process.env.OBJECTTYPEN_TOKEN ?? '0123456789abcdef0123456789abcdef01234567';
|
|
|
|
/**
|
|
* The RegisterRecord objects Objecten holds for a registration reference.
|
|
*
|
|
* The objecttype is resolved by name rather than pinned: Objecttypen echoes the request Host into
|
|
* the objecttype `url`, and Objecten only accepts the one matching its configured api_root — so
|
|
* both must be reached by service name, exactly as the ACL reaches them (ADR-0028).
|
|
*/
|
|
async function registerRecordsFor(reference: string): Promise<Record<string, string>[]> {
|
|
const api = await request.newContext();
|
|
try {
|
|
const types = await api.get(`${OBJECTTYPEN}/api/v2/objecttypes`, {
|
|
headers: { Authorization: `Token ${OBJECTTYPEN_TOKEN}` },
|
|
});
|
|
expect(types.ok(), `Objecttypen returned ${types.status()}`).toBeTruthy();
|
|
const objecttype = ((await types.json()).results as { url: string; name: string }[]).find(
|
|
(o) => o.name === 'RegisterRecord',
|
|
);
|
|
if (!objecttype) throw new Error('the RegisterRecord objecttype is not registered in Objecttypen');
|
|
|
|
const objects = await api.get(`${OBJECTEN}/api/v2/objects`, {
|
|
headers: { Authorization: `Token ${OBJECTEN_TOKEN}`, 'Accept-Crs': 'EPSG:4326' },
|
|
params: { type: objecttype.url, data_attrs: `reference__exact__${reference}` },
|
|
});
|
|
expect(objects.ok(), `Objecten returned ${objects.status()}: ${await objects.text()}`).toBeTruthy();
|
|
return ((await objects.json()).results as { record: { data: Record<string, string> } }[]).map(
|
|
(o) => o.record.data,
|
|
);
|
|
} finally {
|
|
await api.dispose();
|
|
}
|
|
}
|