## What & why S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the `RegisterRecord` in Objecten, not from ZGW zaak events.** Until now the subscriber listened on `zaken` and *inferred* register state from case events — a `zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification at all, so every projection made a second hop to the ACL. The register — a fact about a person — was being reconstructed by guessing at the lifecycle of the case that produced it. - The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish). - An Objecten notification carries **no record data**, only the object URL, so the record is read back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as ADR-0028 established. - The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`, `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all gone, and so is the ACL enrichment hop. - **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently drop every submitted registration from the public register, since only approval wrote a record. - `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all. **ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must agree" — is closed: there is one source now. Closes #153 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing tests committed before the implementation — two red/green pairs, ACL side (06c0444→566ef7d) and subscriber side (142ed45→8af09b2). - [x] Refactor commit follows (b496ac9). - [x] Conventional Commits referencing the issue (`refs #153`). - [x] CI green — all six jobs onb30fa66, `verify-stack` end to end including the e2e. - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (`verify-stack`'s bring-up step — see the wait-healthy fix below). - [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md, e2e header comment. - [x] ADR added in `docs/architecture/`. - [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register shows the same two statuses for the same registrations; only where they come from changed. ## Notes for reviewers **The decision I'd most like a second opinion on** is the one the issue didn't settle: what happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice between (a) the ACL also writing on submit, (b) a public register that lists only actual registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling ADR-0028 set out to remove. All three are laid out in ADR-0030. **The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the object URL (the ACL upserts *one object per registration*, so submit and approval notify about the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6. **The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` → `register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning something else, and a rebuild would then have projected that garbage. It also empties both tables: a pre-slice row describes a zaak event the new projector can't reproject, and those registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten if a long-lived environment ever needs it. **`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back produces no projection row — that's the re-source working, not a gap. ## Three fixes CI found, none of them in the projection logic 1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db` the same way. Anchored on the compose replica suffix, which the verify scripts already do. 2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL put this check in the same bind run-domain-check.sh already handles: `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak reflects the request Host into the zaaktype URL and then rejects it on zaak-create when single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`. 3. **Approval arrives as `partial_update`, not `update`** (0dd26a7→b30fa66) — the one real bug in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but names the action `partial_update`, so the projector dropped every approval. Only the e2e could catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that drives a *real* approval. `verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` / `distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace chain being broken. Filed as **#156** rather than absorbed here. **Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin image, so `objecten-celery` is already a single-process worker. Noted in #156. **Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring `objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too broad a ripple to land inside an unrelated slice — worth its own issue. **Known costs, all in the ADR:** submission is now two writes across two modules and eventually consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc` asserts that path.Reviewed-on: #155
167 lines
9.2 KiB
TypeScript
167 lines
9.2 KiB
TypeScript
import { expect, request, test } from '@playwright/test';
|
|
|
|
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
|
|
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
|
|
// appears in the openbaar register as INGEDIEND; the citizen supplies the documents the process is
|
|
// waiting for (S-10a); a behandelaar then logs in to the behandel portal, finds the registration in
|
|
// the werkbak, and approves it (goedkeuren); the decision completes the Flowable Beoordelen task and
|
|
// flows via the ACL → Objecten → NRC → event-subscriber → projection, and the openbaar register
|
|
// shows INGESCHREVEN.
|
|
//
|
|
// Since ADR-0030 both public statuses come from the register in Objecten, not from ZGW zaak events:
|
|
// the ACL writes the record on submit (INGEDIEND) and upserts it on approval (INGESCHREVEN), so the
|
|
// INGEDIEND assertion below is itself proof of the re-sourced path.
|
|
test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt → public INGESCHREVEN', async ({
|
|
page,
|
|
context,
|
|
}) => {
|
|
// Visiting the guarded page redirects to the Keycloak (mock DigiD) login.
|
|
await page.goto('/');
|
|
|
|
// Keycloak's default login form (stable ids across themes). Its own DigiD user: the verify-* API
|
|
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
|
|
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
|
|
// uses a dedicated citizen no other actor touches.
|
|
await page.locator('#username').fill('emma-burger');
|
|
await page.locator('#password').fill('test123');
|
|
await page.locator('#kc-login').click();
|
|
|
|
// Back on the portal, authenticated.
|
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
|
|
|
await page.getByRole('button', { name: /indienen/i }).click();
|
|
|
|
// The BFF accepted it and the page shows the confirmation with the reference.
|
|
const confirmation = page.getByText(/ontvangen/i);
|
|
await expect(confirmation).toBeVisible();
|
|
const reference = (await confirmation.textContent())?.match(/Referentie:\s*([0-9a-fA-F-]+)/)?.[1];
|
|
expect(reference, 'the confirmation shows a registration reference').toBeTruthy();
|
|
|
|
// The openbaar register (anonymous, its own origin) shows the submitted entry once the projection
|
|
// catches up. We check it on a SEPARATE page so the self-service tab keeps its (in-memory) submitted
|
|
// state — the "Documenten aanleveren" action below acts on that same session. The projection updates
|
|
// asynchronously (NRC → event-subscriber), so reload until *this* submission's row appears. We poll
|
|
// on the reference cell (not a generic INGEDIEND cell): the shared verify stack already holds
|
|
// INGEDIEND rows from earlier checks, so a status-only poll would short-circuit on a stale row.
|
|
const staff = await context.newPage();
|
|
await staff.goto('http://openbaar/');
|
|
await expect(staff.getByRole('heading', { name: /Openbaar BIG-register/i })).toBeVisible();
|
|
|
|
// #78: the reference shown in the public register must be the exact one the citizen saw on the
|
|
// submit confirmation — no mismatch between the two portals.
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('cell', { name: reference }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
await expect(staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGEDIEND' }))
|
|
.toBeVisible();
|
|
|
|
// Provide the documents the registration is waiting for (S-10a), on the still-open self-service tab.
|
|
// The process parks at WachtOpDocumenten only after the zaak is opened; the INGEDIEND row above proves
|
|
// the zaak exists — so the OpenZaak worker has completed and the process is now at the wait — which is
|
|
// why we supply the documents here rather than right after submit, when the trigger would race the
|
|
// wait and no-op. (S-10b turns this into a real file upload; here it is the trigger that unblocks
|
|
// beoordeling.)
|
|
await page.setInputFiles('#diploma', {
|
|
name: 'diploma.pdf',
|
|
mimeType: 'application/pdf',
|
|
buffer: Buffer.from('%PDF-1.4 synthetic diploma\n'),
|
|
});
|
|
await page.getByRole('button', { name: /documenten aanleveren/i }).click();
|
|
await expect(page.getByText(/documenten zijn aangeleverd/i)).toBeVisible();
|
|
|
|
// A behandelaar picks the registration up in the behandel-portal werkbak and approves it (goedkeuren)
|
|
// — the S-12 flow that replaces the temporary admin endpoint. The staff tab switches to the
|
|
// medewerker realm (a different Keycloak realm than the citizen's digid session).
|
|
await staff.goto('http://behandel/');
|
|
await staff.locator('#username').fill('merel-behandelaar');
|
|
await staff.locator('#password').fill('test123');
|
|
await staff.locator('#kc-login').click();
|
|
|
|
await expect(staff.getByRole('heading', { name: /Werkbak/i })).toBeVisible();
|
|
|
|
// The registration reaches the Beoordelen user task only after its documents are provided (above), so
|
|
// it appears in the werkbak asynchronously — reload until this reference's row shows up. Target the
|
|
// decide button by reference (not a generic "Goedkeuren"): the shared verify stack holds other open
|
|
// tasks, so a positional match could act on someone else's registration.
|
|
const goedkeuren = staff.getByRole('button', { name: `Goedkeuren ${reference}` });
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return goedkeuren.count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
|
|
// Click and wait for the decide POST to finish (204) BEFORE leaving the page. `click()` only
|
|
// dispatches the request; navigating away immediately cancels it in flight (nginx logs a 499) and
|
|
// the decision never reaches the domain — so the registration would stay INGEDIEND.
|
|
const decided = staff.waitForResponse(
|
|
(r) =>
|
|
r.url().includes(`/behandel/registrations/${reference}/decide`) &&
|
|
r.request().method() === 'POST',
|
|
);
|
|
await goedkeuren.click();
|
|
expect((await decided).status()).toBe(204);
|
|
|
|
// The approval flows back to the projection; back on the openbaar register *our* row (matched by
|
|
// its reference) now shows INGESCHREVEN.
|
|
await staff.goto('http://openbaar/');
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGESCHREVEN' }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
|
|
// S-19a: the same approval also wrote the canonical register record to Objecten (ADR-0028).
|
|
// Asserted here rather than in verify-domain because this is the only check that drives a *real*
|
|
// approval — verify-domain completes the Beoordelen task straight through Flowable REST, which
|
|
// bypasses the domain `decide` path that calls the ACL.
|
|
const records = await registerRecordsFor(reference);
|
|
// Matched on OUR reference: the verify stack is shared and holds records from earlier checks.
|
|
expect(records, `expected exactly one RegisterRecord for ${reference}`).toHaveLength(1);
|
|
expect(records[0].status).toBe('INGESCHREVEN');
|
|
// The register is world-readable, so the record must carry nothing but the public-safe fields
|
|
// (ADR-0027) — Objecten's own schema validation enforces this, and this proves it end to end.
|
|
expect(Object.keys(records[0]).sort()).toEqual(['id', 'reference', 'status']);
|
|
});
|
|
|
|
const OBJECTEN = process.env.OBJECTEN_URL ?? 'http://objecten:8000';
|
|
const OBJECTTYPEN = process.env.OBJECTTYPEN_URL ?? 'http://objecttypen:8000';
|
|
const OBJECTEN_TOKEN = process.env.OBJECTEN_TOKEN ?? '1234567890abcdef1234567890abcdef12345678';
|
|
const OBJECTTYPEN_TOKEN = process.env.OBJECTTYPEN_TOKEN ?? '0123456789abcdef0123456789abcdef01234567';
|
|
|
|
/**
|
|
* The RegisterRecord objects Objecten holds for a registration reference.
|
|
*
|
|
* The objecttype is resolved by name rather than pinned: Objecttypen echoes the request Host into
|
|
* the objecttype `url`, and Objecten only accepts the one matching its configured api_root — so
|
|
* both must be reached by service name, exactly as the ACL reaches them (ADR-0028).
|
|
*/
|
|
async function registerRecordsFor(reference: string): Promise<Record<string, string>[]> {
|
|
const api = await request.newContext();
|
|
try {
|
|
const types = await api.get(`${OBJECTTYPEN}/api/v2/objecttypes`, {
|
|
headers: { Authorization: `Token ${OBJECTTYPEN_TOKEN}` },
|
|
});
|
|
expect(types.ok(), `Objecttypen returned ${types.status()}`).toBeTruthy();
|
|
const objecttype = ((await types.json()).results as { url: string; name: string }[]).find(
|
|
(o) => o.name === 'RegisterRecord',
|
|
);
|
|
if (!objecttype) throw new Error('the RegisterRecord objecttype is not registered in Objecttypen');
|
|
|
|
const objects = await api.get(`${OBJECTEN}/api/v2/objects`, {
|
|
headers: { Authorization: `Token ${OBJECTEN_TOKEN}`, 'Accept-Crs': 'EPSG:4326' },
|
|
params: { type: objecttype.url, data_attrs: `reference__exact__${reference}` },
|
|
});
|
|
expect(objects.ok(), `Objecten returned ${objects.status()}: ${await objects.text()}`).toBeTruthy();
|
|
return ((await objects.json()).results as { record: { data: Record<string, string> } }[]).map(
|
|
(o) => o.record.data,
|
|
);
|
|
} finally {
|
|
await api.dispose();
|
|
}
|
|
}
|