# Gitea Actions gotchas How our CI (Gitea Actions on the hosted **`ubuntu-latest`** runner) differs from a local run, and the workarounds in this repo. Referenced by `CLAUDE.md` §8.7/§15. **One root cause sits under most of this:** the runner executes the job **inside a container**, so when a step runs `docker compose up`, Compose starts the stack as **sibling containers** on the host's daemon. Anything that assumes the job and those containers share a filesystem — or a `localhost` — breaks. | Gotcha | Fix | Lives in | |---|---|---| | Bind-mounted config arrives empty | `docker cp` config into external volumes | `infra/seed-config.sh` | | `docker compose up --wait` is unsupported / flaky | poll health with `docker inspect` | `infra/wait-healthy.sh` | | `pg_isready` passes before PostGIS is ready | add a `PostGIS_Version()` probe | the db healthchecks | | `upload-artifact@v4` fails ("not supported on GHES") | pin `@v3` | `.gitea/workflows/ci.yaml` (`mutation` job) | | `upload-artifact@v3` fails with "Artifact service responded with 500" | mark the upload `continue-on-error: true` (server-side; issue #62) | `.gitea/workflows/ci.yaml` (`mutation` job) | --- ## 1. Bind mounts don't reach the containers **Symptom** — green locally, but `compose-smoke` fails with: ``` oz-init-1 | CommandError: Yaml file `/app/setup_configuration/data.yaml` does not exist. ``` Migrations run fine; only the step that reads a *mounted* file fails. The same trap hits `nrc-init`, `flowable-init`, and `keycloak`. **Why** — a relative bind mount like `./openzaak/setup_configuration:/app/...` is resolved by Compose to a path *inside the job container* (`/workspace/.../setup_configuration`). The daemon then looks for that path on *its own host*, doesn't find it, and mounts an **empty directory**. (It works on a runner that executes jobs on the host — which is why moving to `ubuntu-latest` exposed it.) **Fix** — use the upstream images verbatim (no build) and stream config into **external named volumes** with `docker cp`, which copies over the Docker API and so works wherever the daemon runs. `infra/seed-config.sh` creates each volume, mounts it in a throwaway helper, and copies the files in: | Asset | Volume | Mounted at | |---|---|---| | OpenZaak `data.yaml` | `rr-oz-config` | `oz-init:/app/setup_configuration` | | Keycloak realms | `rr-kc-realms` | `keycloak:/opt/keycloak/data/import` | | `registratie.bpmn` | `rr-fl-bpmn` | `flowable-init:/work` | The volumes are `external: true` with fixed names, so they resolve identically under docker compose and podman-compose. `make` seeds before every `up`; `make down` removes them. (Open Notificaties needs nothing — `nrc-init` migrates only.) **Consequence — bare `docker compose up` can't self-seed external volumes:** - **CI / Linux / macOS:** `make up` or `make smoke` (seed, then start). - **No-make / Windows:** `infra/docker-compose.local.yml` — a twin stack that **bind-mounts** the config instead. Bind mounts are fine *locally* because a local daemon can see your working directory, so `docker compose -f infra/docker-compose.local.yml up -d` just works. **Why not the obvious alternatives** - *Bake config into an image* (incl. an inline Dockerfile) — `docker compose up` would then work unaided, but it's a build; we wanted the upstream images as-is. - *Compose `configs:` with inline `content`* — Compose writes a client-side temp file and bind-mounts it, hitting the exact same problem. - *A host-executing runner* — bind mounts would work with zero seeding, but it reintroduces a self-hosted runner and undoes the move to `ubuntu-latest`. --- ## 2. Readiness: poll health, don't use `--wait` `docker compose up --wait` looks ideal but fails us three ways: - **podman-compose doesn't implement it** (`unrecognized arguments: --wait`) — so it would break local dev. - A project-wide `--wait` **treats a one-shot exiting `0` as a failure** unless something `depends_on` it with `service_completed_successfully`. `flowable-init` deploys the BPMN and exits with no dependant, so `--wait` fails the moment it does — last line `container infra-flowable-init-1 exited (0)`. - The containerized runner **can't reach published host ports**, so an external `curl localhost:8080/health` can't work either. **Fix** — `infra/wait-healthy.sh` polls each durable service (`openzaak nrc-web acl bff`, listed as `WAIT_SVCS` in the `Makefile`) with `docker ps` + `docker inspect '{{.State.Health.Status}}'` until it reports `healthy`. It uses only primitives both runtimes support, reads the **in-container** healthcheck (no host port needed), and ignores the one-shots (they only need to have run). `WAIT_TIMEOUT` defaults to 420 s — enough for the cold OpenZaak migrate (~90 s) plus app start. --- ## 3. `pg_isready` passes before PostGIS is ready `pg_isready` succeeds as soon as the TCP port is open — *before* the `postgis/postgis` image has finished running `CREATE EXTENSION postgis`. An init container that starts migrating in that window can fail on a missing PostGIS. So the db healthchecks add a `SELECT PostGIS_Version()` probe, making dependents wait for the extension, not just the port. --- ## 4. `actions/upload-artifact@v4` refuses to run on Gitea **Symptom** — the `mutation` job's `make mutation` step passes (95% score), but the upload step right after it fails the job: ``` ::error::@actions/artifact v2.0.0+, upload-artifact@v4+ and download-artifact@v4+ are not currently supported on GHES. ❌ Failure - Main https://github.com/actions/upload-artifact@v4 ``` **Why** — `upload-artifact@v4` bundles `@actions/artifact` v2, which inspects the server URL and **hard-aborts on anything that isn't `github.com`**, treating Gitea as an unsupported GitHub Enterprise Server. The check fires regardless of whether the Gitea server can actually store artifacts (1.24+ can). It is the *action*, not the server, that refuses. **Fix** — pin **`actions/upload-artifact@v3`** (and `download-artifact@v3` if ever needed). v3 uses the older artifact protocol that Gitea implements, and has no GHES guard. Inputs are the same (`name`, `path`, `if-no-files-found`), so it is a drop-in swap. Do **not** bump to `@v4` until act_runner advertises github.com-compatible artifact support. **Second failure mode — the server's artifact backend returns 500.** Even on the correctly-pinned `@v3`, uploads can fail with: ``` Create Artifact Container - Attempt 5 of 5 failed with error: Artifact service responded with 500 ::error::Create Artifact Container failed: Artifact service responded with 500 ``` This is the **Gitea server's** artifact storage failing (not the action's GHES guard), so it is outside the repo's control. Because the `mutation` job's upload steps run with `if: always()`, that 500 would fail the job even though the ratchet passed. **Fix:** mark the uploads `continue-on-error: true` (issue #62). The mutation *gate* is the Stryker ratchet — `make mutation`'s exit code fails the job on a real regression — so the report upload is best-effort: when the server's artifact storage is restored, reports publish again with no workflow change. --- ## 5. A runner process can't reach a service container's published port **Symptom** — green locally, but a CI step that runs *on the runner* and talks to a compose service over `localhost` fails. The ACL integration test's seed died with: ``` OpenZaak ready (000) urllib.error.URLError: make: *** [Makefile:114: integration] Error 1 ``` OpenZaak was demonstrably up — uwsgi had been serving for ~2 minutes — yet `curl`/`urllib` to `localhost:8000` from the runner were refused the whole time. **Why** — the same sibling-container split as §1. Compose starts the stack via the host daemon, so `ports: ["8000:8000"]` publishes to the *daemon host*, not to the job container. From the runner, `localhost:8000` has nothing listening. (`make smoke` sidesteps this by polling readiness via `docker inspect` (§2), never a service port.) **Fix** — don't talk to service ports from the runner. Either check state via `docker inspect` (health), or run the client **inside the compose network** so it reaches the service by name (`http://openzaak:8000`). For a test/seed that needs the repo's own code, deliver it via a **built image** (not a bind mount — §1), then `docker run --network _cg …`. **Applied** — `make integration` (ADR-0006) and `make verify-notifications` (ADR-0007) do exactly this: they run the seed/test/driver as containers on the stack network and reach services by **container IP** (see §6). --- ## 6. OpenZaak / NRC reject single-label hosts in URLs **Symptom** — talking to OpenZaak or NRC by compose **service name** fails where a URL is validated: catalogus/zaaktype filters, the zaak `zaaktype` URL, and abonnement `callbackUrl` come back `400 "Voer een geldige URL in."` — even though the host resolves and is reachable. **Why** — these apps validate URLs with Django's `URLValidator`, which rejects a **single-label** host like `openzaak` or `nrc-web` (no dot, and not `localhost`). `localhost` passes (so it's invisible in host-port-based local runs); in-network the reality is a service name or an IPv4 literal — and only the IP passes. **Fix** — in-network tooling reaches OpenZaak/NRC by **container IP** (`docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'`), not service name; the notif verify harness also registers the sink callback by IP. (`infra/run-acl-integration.sh`, `infra/run-notification-check.sh`.) **Related — abonnement callbacks must enforce auth.** NRC probes a callback when an abonnement is registered and refuses it (`no-auth-on-callback-url`) unless it returns **401** without the configured `Authorization`. The verify sink (`infra/notification-sink.py`) enforces a bearer token for exactly this reason. --- ## 7. A job with `if: ${{ !cancelled() }}` (or `always()`) + `needs` sticks in "waiting" **Symptom** — after upgrading to **Gitea 1.27** + **act_runner 2.0.0**, one job never starts: the run sits in state `waiting` forever, the job has **no logs** (never dispatched to a runner), and the other jobs finish normally. `main` stays pending/red. Seen on the `verify-stack` job (#134). **Why** — Gitea 1.27 reworked cancellation/aggregation: a job gated by a **status-function `if`** (`always()` / `cancelled()` / `!cancelled()`) on top of `needs` now routes through a new transitional **`Cancelling`** job state plus a server↔runner **capability negotiation** ("Requires Gitea Runner 2.0.0"). On the 1.27 + 2.0.0 pairing that handshake doesn't resolve for such a job, so it's never offered to a runner and never leaves `waiting`. Jobs with no `if`/`needs` are unaffected. (Related upstream: go-gitea/gitea#31074, #27116, #35782.) **Fix** — don't gate a `needs` job with a status-function `if`. Use the default `if: success()` (i.e. omit the `if`). If you need "run even when an upstream job fails", prefer serialising with a `concurrency` group over `needs` + `always()`. **Also** — a run already stuck this way will **not** clear itself; force-cancel it from the Actions UI (plain cancel can also stall on this version, #35782). Push the workflow fix to produce a fresh run. --- ## 8. Job summaries (`$GITHUB_STEP_SUMMARY`) need Gitea ≥1.27 + runner ≥2.0 Markdown a step appends to the `$GITHUB_STEP_SUMMARY` file renders on the run page (no artifact download). We use it for per-run reports (#136): mutation scores (Stryker `markdown` reporter), per-service unit results (`infra/trx-summary.py` over TRX), per-frontend results (`infra/vitest-summary.py` over each app's vitest JSON), the verify-stack check table, and per-spec e2e results (`infra/playwright-summary.py`). **Requirements / conventions:** - Requires **Gitea ≥ 1.27** (stores/renders summaries) and **act_runner ≥ 2.0.0** (uploads them). Older pairings silently skip the upload. - **Guard every write:** `[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0` — on a runner without support the var is unset and `>> "$GITHUB_STEP_SUMMARY"` would be an ambiguous-redirect error. The guard makes the step a no-op locally / on old runners. - Use `if: always()` (step-level) on summary steps so they render even when the thing they report on failed. Step-level `always()` is fine on 2.0.0 — unlike the *job*-level status-function `if` of §7. - Getting a report out of the e2e container: Playwright writes `playwright-report.json` inside the container; `infra/run-e2e-check.sh` `docker cp`s it back to the host (capturing the test exit code first) so the summary step can read it. --- ## 9. `if: always()` does not survive the job being killed — bound the work itself `if: always()` makes a step run when an *earlier step failed*. It does **not** help when the job as a whole is stopped: the run's remaining steps are simply never dispatched. That is how #161 lost its diagnosis. `verify-stack` entered `make verify-e2e` at 09:48:17 and the job ended at 10:14:54 — 26½ minutes later, mid-suite. Every step after the e2e shows a **0-second `failure`** stamped at that same instant: ``` 14 failure 09:48:17 -> 10:14:54 Self-service e2e (Playwright, login → submit → success) 15 failure 10:14:54 -> 10:14:54 verify-stack check summary ← if: always() 16 failure 10:14:54 -> 10:14:54 e2e spec summary ← if: always() 17 failure 10:14:54 -> 10:14:54 Dump container logs on failure ← if: failure() 18 failure 10:14:54 -> 10:14:54 Tear down ← if: always() ``` So the per-spec summary, the container-log dump and the teardown never ran, and the job log — which also loses whatever the killed process had buffered — ended at a single `✘` line. A job that dies takes its own post-mortem with it. **Read the step timings, not just the log.** `GET /api/v1/repos/{owner}/{repo}/actions/jobs/{id}` returns every step with `started_at`/`completed_at`; a row of identical zero-length steps at the end means *killed*, not *silent*. (Job ids come from `…/actions/runs/{run}/jobs`, and that route returns only the **latest attempt** — a re-run hides the failed one, so keep the failing job id from the original report. Logs: `…/actions/jobs/{id}/logs`, see also `gitea-ci-logs`.) **Conventions that follow:** - **Bound long-running work inside the tool**, where it can still report. Playwright's `globalTimeout` (`tests/e2e/playwright.config.ts`) ends the run, writes the JSON report and exits, so the summary and log-dump steps still get their turn. A `timeout-minutes` on the job would reproduce the very failure above. - **Never let an auto-waiting action be the timeout.** Playwright actions (`fill`, `click`) inherit the *test* timeout, not `expect.timeout`, so a missing element costs the full 90 s and reports `locator.fill: Test timeout …` — the symptom. Assert the element visible first with its own budget and a message (`tests/e2e/keycloak-login.ts`). - Remember `concurrency.cancel-in-progress: true` in `ci.yaml`: a new push to the same ref, or a re-run, kills the in-flight run the same way. Check `run_attempt` before concluding a job hung.