fix(e2e): fail fast when a portal never reaches Keycloak, and bound the run (refs #161)

Two defects behind #161's opaque 36-minute verify-stack job.

**A login that never gets its form ate the test timeout.** `fill()` auto-waits
until the *test* timeout (90s), not the 15s expect timeout, so a portal that
serves its page but never bootstraps — its config.json fetch or the OIDC
discovery behind `authorize()` failed, and main.ts only console.errors — spent
90 seconds to report `locator.fill: Test timeout of 90000ms exceeded`: the
symptom, not the cause. That is catalogus.spec's 1.8 minutes in the issue.
Both Keycloak forms are now asserted visible first, with a 20s budget and a
message naming the step that never happened.

Verified against a real blank-bootstrap portal (a beheer image served with a
config.json that is not JSON): fails in 20.2s with
"the Keycloak login form never appeared — the portal did not reach Keycloak
(check its config.json fetch and the OIDC discovery …)".

**A wedged suite consumed the job.** Nothing bounded the run, so CI killed the
job — and with it the `if: always()` steps that would have explained the
failure: neither the per-spec summary nor the container-log dump ran (both show
0-second failures at the kill in run 739's metadata). `globalTimeout` makes
Playwright stop and *report* instead, so the JSON report is written and those
steps still run. 12 minutes over a ~1-minute suite: a backstop, not a budget.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
not
2026-09-04 11:46:28 +02:00
co-authored by Claude Opus 5
parent 27f2607e4e
commit e7d4ed8ad4
2 changed files with 35 additions and 4 deletions
+29 -4
View File
@@ -2,7 +2,7 @@ import { createHmac } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { Page } from '@playwright/test';
import { expect, type Page } from '@playwright/test';
// Every portal login in the suite goes through this module — citizen realms (mock DigiD) and the
// medewerker realm alike — so the shared Keycloak form handling lives in exactly one place.
@@ -14,6 +14,18 @@ const OTP_SECRET = 'BIGMEDEWERKEROTPSEED';
export const OTP_PERIOD_MS = 30_000;
/**
* How long a Keycloak form gets to appear. Generous enough for a cold first browser launch and a
* loaded stack, far short of the 90-second test timeout an auto-waiting action would otherwise eat.
*/
const FORM_TIMEOUT_MS = 20_000;
const FORM_NEVER_APPEARED =
'the Keycloak login form never appeared — the portal did not reach Keycloak (check its ' +
'config.json fetch and the OIDC discovery on the authority it was built with)';
const OTP_NEVER_APPEARED =
'the Keycloak OTP form never appeared — the password step did not complete (check the ' +
'medewerker realm seeded this user with both a password and a TOTP credential)';
// RFC 6238 TOTP: HMAC-SHA1 over the 30-second counter, dynamically truncated to 6 digits.
export function totp(secret = OTP_SECRET, at = Date.now()): string {
const counter = Buffer.alloc(8);
@@ -48,8 +60,17 @@ function spendCounter(username: string): number {
/**
* Fill Keycloak's login form. Every portal is guarded, so the first navigation redirects here; the
* form ids are stable across themes.
*
* The form is asserted visible *before* it is filled. A portal that never reaches Keycloak — its
* runtime `config.json` fetch or the OIDC discovery behind `authorize()` failed, so it never
* bootstrapped and shows a blank page (main.ts only logs to the console) — would otherwise leave
* `fill()` auto-waiting until the whole test times out: 90 seconds spent to report
* `locator.fill: Test timeout of 90000ms exceeded`, naming the symptom and not the cause. That is
* how #161's catalogus.spec burned 1.8 minutes. This fails in a quarter of the time and says which
* step never happened.
*/
async function submitPassword(page: Page, username: string): Promise<void> {
await expect(page.locator('#username'), FORM_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
await page.locator('#username').fill(username);
await page.locator('#password').fill('test123');
await page.locator('#kc-login').click();
@@ -64,9 +85,13 @@ export async function loginBurger(page: Page, username: string): Promise<void> {
export async function loginMedewerker(page: Page, username: string): Promise<void> {
await submitPassword(page, username);
// Keycloak's conditional-OTP step. Wait out the rest of the window if the counter we may spend is
// still in the future; its lookAheadWindow would accept the code a moment early, but only by one
// counter — waiting keeps a third login in the same window valid too.
// Keycloak's conditional-OTP step. Same reasoning as the password form above: assert it arrived
// rather than letting `fill()` swallow the test timeout.
await expect(page.locator('#otp'), OTP_NEVER_APPEARED).toBeVisible({ timeout: FORM_TIMEOUT_MS });
// Wait out the rest of the window if the counter we may spend is still in the future; Keycloak's
// lookAheadWindow would accept the code a moment early, but only by one counter — waiting keeps a
// third login in the same window valid too.
const counter = spendCounter(username);
await page.waitForTimeout(Math.max(0, counter * OTP_PERIOD_MS - Date.now()));
await page.locator('#otp').fill(totp(OTP_SECRET, counter * OTP_PERIOD_MS));
+6
View File
@@ -15,6 +15,12 @@ export default defineConfig({
timeout: 90_000,
expect: { timeout: 15_000 },
retries: 1,
// Bound the whole run, not just each test (#161). A wedged suite used to run until CI killed the
// job — which also killed the `if: always()` steps that would have said why: the per-spec summary
// and the container-log dump never ran, leaving a 36-minute job whose entire surviving output was
// one ✘ line. On `globalTimeout` Playwright stops and *reports*, so the JSON report is written and
// those steps still run. Generous over the ~1-minute suite: this is a backstop, not a budget.
globalTimeout: 12 * 60_000,
// Run the specs serially. Each spec drives a full `channel: 'chromium'` browser, and the e2e
// shares an 8 GB runner with the entire compose stack (OpenZaak, NRC, Keycloak, Flowable, 4×
// Postgres, every service + 3 portals). Two parallel browsers exhaust memory and the renderer is