diff --git a/docs/runbooks/gitea-actions-gotchas.md b/docs/runbooks/gitea-actions-gotchas.md index 8ecdfc0..e41c5cd 100644 --- a/docs/runbooks/gitea-actions-gotchas.md +++ b/docs/runbooks/gitea-actions-gotchas.md @@ -245,3 +245,47 @@ the verify-stack check table, and per-spec e2e results (`infra/playwright-summar - Getting a report out of the e2e container: Playwright writes `playwright-report.json` inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host (capturing the test exit code first) so the summary step can read it. + +--- + +## 9. `if: always()` does not survive the job being killed — bound the work itself + +`if: always()` makes a step run when an *earlier step failed*. It does **not** help when +the job as a whole is stopped: the run's remaining steps are simply never dispatched. + +That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17 +and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e +shows a **0-second `failure`** stamped at that same instant: + +``` +14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success) +15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always() +16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always() +17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure() +18 failure 10:14:54 -> 10:14:54 Tear down ← if: always() +``` + +So the per-spec summary, the container-log dump and the teardown never ran, and the job +log — which also loses whatever the killed process had buffered — ended at a single `✘` +line. A job that dies takes its own post-mortem with it. + +**Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}` +returns every step with `started_at`/`completed_at`; a row of identical zero-length +steps at the end means *killed*, not *silent*. (Job ids come from +`…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a +re-run hides the failed one, so keep the failing job id from the original report. Logs: +`…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.) + +**Conventions that follow:** + +- **Bound long-running work inside the tool**, where it can still report. Playwright's + `globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON + report and exits, so the summary and log-dump steps still get their turn. A + `timeout-minutes` on the job would reproduce the very failure above. +- **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`, + `click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs + the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the + element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`). +- Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same + ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before + concluding a job hung.