## What & why #161 is really two defects, and the second one is why the first was undiagnosable. **A wedged suite consumed the job, and took the post-mortem with it.** Nothing bounded the Playwright run, so CI stopped the job mid-suite — and `if: always()` does not survive that. Run 739's job metadata shows every step after the e2e as a **0-second failure** stamped at the kill: ``` 14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright …) 15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always() 16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always() 17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure() 18 failure 10:14:54 -> 10:14:54 Tear down ← if: always() ``` So the per-spec summary, the container-log dump and the teardown never ran, and the log lost whatever the killed process had buffered — leaving the single `✘` line the issue was filed from. `globalTimeout` now makes Playwright stop and *report*: the JSON report is written and those steps still get their turn. (A `timeout-minutes` on the job would have reproduced the same failure, so there isn't one.) The "~24-minute gap" is that kill, not necessarily a hang — note run 739 shows `run_attempt: 2`, and `concurrency.cancel-in-progress` kills an in-flight run on any re-run or push. **A login that never got its form ate the 90-second test timeout.** Playwright actions auto-wait until the *test* timeout, not `expect.timeout` — so a portal that serves its page but never bootstraps (its `config.json` fetch or the OIDC discovery behind `authorize()` failed; `main.ts` only `console.error`s) spent 90s to report `locator.fill: Test timeout of 90000ms exceeded`: the symptom, not the cause. That is catalogus.spec's 1.8 minutes. Both Keycloak forms are now asserted visible first, with a 20s budget and a message naming the step that never happened. Verified against a real blank-bootstrap portal — the beheer image served with a `config.json` that is not JSON — which fails in **20.2s** with *"the Keycloak login form never appeared — the portal did not reach Keycloak (check its config.json fetch and the OIDC discovery …)"*. **And the summary now says why.** The per-spec table (#136) rendered a verdict icon and nothing else, so even a surviving summary cost a log dive. Failing specs now carry their first error, flattened for a table cell (ANSI stripped, newlines collapsed, `|` escaped, clipped) — shape verified against a real @playwright/test 1.61 failing report, with a stdlib assert self-check on `make unit`. Closes #161 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation. - [x] Implementation makes the test pass; refactor commit follows (login helper dedup). - [x] Conventional Commits referencing the issue (`refs #161`). - [ ] CI green — all Gitea Actions jobs. - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (untouched). - [x] Docs updated — `docs/runbooks/gitea-actions-gotchas.md` §9. - [x] ADR — not needed: no boundary, dependency or coupling rule touched (test/CI infra only). - [x] Demo note — not applicable: nothing user-visible. ## Notes for reviewers **What this does not do: identify why the beheerder login failed that once.** The evidence to do that was destroyed by defect 2, which is what this PR fixes. The suite ran green here five times today (catalogus.spec 1.1–5.3s each) — but a local box is not the loaded CI runner, so that is weak evidence and I am not claiming the flake is gone. What changes is that the next occurrence is bounded and self-describing: it fails in 20s naming the failing step, the JSON report survives, and the summary prints the error. Please keep #161 in mind rather than treating this as proof. **Two follow-ups I did not pull into this PR:** - *All four portals show a permanently blank page if their startup fetch fails* — `main.ts` does `fetch('config.json').then(bootstrap).catch(console.error)`, one shot, no UI and no recovery. That is a real product gap (the deliberately-broken portal above is exactly what a user would see) and wants its own slice, not a test-infra PR. - `retries: 1` is untouched. CLAUDE.md §15 says flaky tests are fixed rather than retried, but removing retries while a real flake is unexplained would trade a rare red for a frequent one. Worth revisiting once #161 recurs (or doesn't) with the new diagnostics. The login-helper rename (`medewerker-login.ts` → `keycloak-login.ts`, citizen logins routed through `loginBurger`) is its own no-behaviour-change commit: the three citizen specs each duplicated the same three-line login, so guarding the login path once meant routing them through it first.Reviewed-on: #165
170 lines
9.6 KiB
TypeScript
170 lines
9.6 KiB
TypeScript
import { expect, request, test } from '@playwright/test';
|
|
import { loginBurger, loginMedewerker } from './keycloak-login';
|
|
|
|
// Walking-skeleton happy path (S-08d + S-09 + S-09b + S-12 + S-10a + S-19b-2): a zorgprofessional
|
|
// logs in via mock DigiD and submits through the self-service portal → BFF → domain; the entry
|
|
// appears in the openbaar register as INGEDIEND; the citizen supplies the documents the process is
|
|
// waiting for (S-10a); a behandelaar then logs in to the behandel portal, finds the registration in
|
|
// the werkbak, and approves it (goedkeuren); the decision completes the Flowable Beoordelen task and
|
|
// flows via the ACL → Objecten → NRC → event-subscriber → projection, and the openbaar register
|
|
// shows INGESCHREVEN.
|
|
//
|
|
// Since ADR-0030 both public statuses come from the register in Objecten, not from ZGW zaak events:
|
|
// the ACL writes the record on submit (INGEDIEND) and upserts it on approval (INGESCHREVEN), so the
|
|
// INGEDIEND assertion below is itself proof of the re-sourced path.
|
|
test('DigiD submit → public INGEDIEND → documenten → behandelaar goedkeurt → public INGESCHREVEN', async ({
|
|
page,
|
|
context,
|
|
}) => {
|
|
// Visiting the guarded page redirects to the Keycloak (mock DigiD) login.
|
|
await page.goto('/');
|
|
|
|
// Keycloak's default login form (stable ids across themes). Its own DigiD user: the verify-* API
|
|
// checks submit as jan-burger (bsn 123456782) before the e2e runs on the shared stack, and
|
|
// resume-on-load (S-26) would otherwise restore one of those on login — so each self-service spec
|
|
// uses a dedicated citizen no other actor touches.
|
|
await loginBurger(page, 'emma-burger');
|
|
|
|
// Back on the portal, authenticated.
|
|
await expect(page.getByRole('heading', { name: /Zelfservice/i })).toBeVisible();
|
|
|
|
await page.getByRole('button', { name: /indienen/i }).click();
|
|
|
|
// The BFF accepted it and the page shows the confirmation with the reference.
|
|
const confirmation = page.getByText(/ontvangen/i);
|
|
await expect(confirmation).toBeVisible();
|
|
const reference = (await confirmation.textContent())?.match(/Referentie:\s*([0-9a-fA-F-]+)/)?.[1];
|
|
expect(reference, 'the confirmation shows a registration reference').toBeTruthy();
|
|
|
|
// The openbaar register (anonymous, its own origin) shows the submitted entry once the projection
|
|
// catches up. We check it on a SEPARATE page so the self-service tab keeps its (in-memory) submitted
|
|
// state — the "Documenten aanleveren" action below acts on that same session. The projection updates
|
|
// asynchronously (NRC → event-subscriber), so reload until *this* submission's row appears. We poll
|
|
// on the reference cell (not a generic INGEDIEND cell): the shared verify stack already holds
|
|
// INGEDIEND rows from earlier checks, so a status-only poll would short-circuit on a stale row.
|
|
const staff = await context.newPage();
|
|
await staff.goto('http://openbaar/');
|
|
await expect(staff.getByRole('heading', { name: /Openbaar BIG-register/i })).toBeVisible();
|
|
|
|
// #78: the reference shown in the public register must be the exact one the citizen saw on the
|
|
// submit confirmation — no mismatch between the two portals.
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('cell', { name: reference }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
await expect(staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGEDIEND' }))
|
|
.toBeVisible();
|
|
|
|
// A behandelaar opens the behandel-portal werkbak and approves the registration (goedkeuren) — the
|
|
// S-12 flow that replaces the temporary admin endpoint. The staff tab switches to the medewerker
|
|
// realm (a different Keycloak realm than the citizen's digid session).
|
|
//
|
|
// The werkbak is opened BEFORE the citizen supplies the documents that route the registration to
|
|
// Beoordelen, so its row cannot be there at page load: the only thing that can deliver it to this
|
|
// already-open page is the werkbak refreshing itself (S-26/#162, ADR-0032). This spec used to
|
|
// `staff.reload()` in a poll loop here; the absence of that reload is the live-refresh assertion.
|
|
await staff.goto('http://behandel/');
|
|
// That realm enforces MFA (S-15c), so the behandelaar logs in with password + TOTP.
|
|
await loginMedewerker(staff, 'merel-behandelaar');
|
|
|
|
await expect(staff.getByRole('heading', { name: /Werkbak/i })).toBeVisible();
|
|
|
|
// Target the decide button by reference (not a generic "Goedkeuren"): the shared verify stack holds
|
|
// other open tasks, so a positional match could act on someone else's registration.
|
|
const goedkeuren = staff.getByRole('button', { name: `Goedkeuren ${reference}` });
|
|
await expect(goedkeuren, 'the registration is not awaiting beoordeling yet').toBeHidden();
|
|
|
|
// Provide the documents the registration is waiting for (S-10a), on the still-open self-service tab.
|
|
// The process parks at WachtOpDocumenten only after the zaak is opened; the INGEDIEND row above proves
|
|
// the zaak exists — so the OpenZaak worker has completed and the process is now at the wait — which is
|
|
// why we supply the documents here rather than right after submit, when the trigger would race the
|
|
// wait and no-op. (S-10b turns this into a real file upload; here it is the trigger that unblocks
|
|
// beoordeling.)
|
|
await page.bringToFront();
|
|
await page.setInputFiles('#diploma', {
|
|
name: 'diploma.pdf',
|
|
mimeType: 'application/pdf',
|
|
buffer: Buffer.from('%PDF-1.4 synthetic diploma\n'),
|
|
});
|
|
await page.getByRole('button', { name: /documenten aanleveren/i }).click();
|
|
await expect(page.getByText(/documenten zijn aangeleverd/i)).toBeVisible();
|
|
|
|
// Back to the werkbak — untouched since login, never reloaded. The row arrives on its own once the
|
|
// DMN routes the registration to Beoordelen. (Foregrounded so Chromium doesn't throttle the page's
|
|
// refresh timer as a hidden tab.)
|
|
await staff.bringToFront();
|
|
await expect(goedkeuren).toBeVisible({ timeout: 30_000 });
|
|
|
|
// Click and wait for the decide POST to finish (204) BEFORE leaving the page. `click()` only
|
|
// dispatches the request; navigating away immediately cancels it in flight (nginx logs a 499) and
|
|
// the decision never reaches the domain — so the registration would stay INGEDIEND.
|
|
const decided = staff.waitForResponse(
|
|
(r) =>
|
|
r.url().includes(`/behandel/registrations/${reference}/decide`) &&
|
|
r.request().method() === 'POST',
|
|
);
|
|
await goedkeuren.click();
|
|
expect((await decided).status()).toBe(204);
|
|
|
|
// The approval flows back to the projection; back on the openbaar register *our* row (matched by
|
|
// its reference) now shows INGESCHREVEN.
|
|
await staff.goto('http://openbaar/');
|
|
await expect
|
|
.poll(async () => {
|
|
await staff.reload();
|
|
return staff.getByRole('row', { name: reference }).getByRole('cell', { name: 'INGESCHREVEN' }).count();
|
|
}, { timeout: 30_000, intervals: [1_000, 2_000, 3_000, 5_000] })
|
|
.toBeGreaterThan(0);
|
|
|
|
// S-19a: the same approval also wrote the canonical register record to Objecten (ADR-0028).
|
|
// Asserted here rather than in verify-domain because this is the only check that drives a *real*
|
|
// approval — verify-domain completes the Beoordelen task straight through Flowable REST, which
|
|
// bypasses the domain `decide` path that calls the ACL.
|
|
const records = await registerRecordsFor(reference);
|
|
// Matched on OUR reference: the verify stack is shared and holds records from earlier checks.
|
|
expect(records, `expected exactly one RegisterRecord for ${reference}`).toHaveLength(1);
|
|
expect(records[0].status).toBe('INGESCHREVEN');
|
|
// The register is world-readable, so the record must carry nothing but the public-safe fields
|
|
// (ADR-0027) — Objecten's own schema validation enforces this, and this proves it end to end.
|
|
expect(Object.keys(records[0]).sort()).toEqual(['id', 'reference', 'status']);
|
|
});
|
|
|
|
const OBJECTEN = process.env.OBJECTEN_URL ?? 'http://objecten:8000';
|
|
const OBJECTTYPEN = process.env.OBJECTTYPEN_URL ?? 'http://objecttypen:8000';
|
|
const OBJECTEN_TOKEN = process.env.OBJECTEN_TOKEN ?? '1234567890abcdef1234567890abcdef12345678';
|
|
const OBJECTTYPEN_TOKEN = process.env.OBJECTTYPEN_TOKEN ?? '0123456789abcdef0123456789abcdef01234567';
|
|
|
|
/**
|
|
* The RegisterRecord objects Objecten holds for a registration reference.
|
|
*
|
|
* The objecttype is resolved by name rather than pinned: Objecttypen echoes the request Host into
|
|
* the objecttype `url`, and Objecten only accepts the one matching its configured api_root — so
|
|
* both must be reached by service name, exactly as the ACL reaches them (ADR-0028).
|
|
*/
|
|
async function registerRecordsFor(reference: string): Promise<Record<string, string>[]> {
|
|
const api = await request.newContext();
|
|
try {
|
|
const types = await api.get(`${OBJECTTYPEN}/api/v2/objecttypes`, {
|
|
headers: { Authorization: `Token ${OBJECTTYPEN_TOKEN}` },
|
|
});
|
|
expect(types.ok(), `Objecttypen returned ${types.status()}`).toBeTruthy();
|
|
const objecttype = ((await types.json()).results as { url: string; name: string }[]).find(
|
|
(o) => o.name === 'RegisterRecord',
|
|
);
|
|
if (!objecttype) throw new Error('the RegisterRecord objecttype is not registered in Objecttypen');
|
|
|
|
const objects = await api.get(`${OBJECTEN}/api/v2/objects`, {
|
|
headers: { Authorization: `Token ${OBJECTEN_TOKEN}`, 'Accept-Crs': 'EPSG:4326' },
|
|
params: { type: objecttype.url, data_attrs: `reference__exact__${reference}` },
|
|
});
|
|
expect(objects.ok(), `Objecten returned ${objects.status()}: ${await objects.text()}`).toBeTruthy();
|
|
return ((await objects.json()).results as { record: { data: Record<string, string> } }[]).map(
|
|
(o) => o.record.data,
|
|
);
|
|
} finally {
|
|
await api.dispose();
|
|
}
|
|
}
|