Files
register-referentie/docs/runbooks/gitea-actions-gotchas.md
Niek Otten a0aa22c80b
All checks were successful
CI / lint (pull_request) Successful in 50s
CI / build (pull_request) Successful in 42s
CI / unit (pull_request) Successful in 50s
CI / compose-smoke (pull_request) Successful in 4m52s
fix(infra): smoke waits on durable services, not the whole project (refs #30)
Run 28 got the full stack healthy but `compose-smoke` still failed. The last
compose line before the error was:

  container infra-flowable-init-1 exited (0)

`docker compose up --wait` treats a service that exits as a failure of the
"stay running" condition unless something depends on it via
`service_completed_successfully`. oz-init/nrc-init are fine (openzaak/nrc-web
depend on them), but flowable-init deploys the BPMN and exits 0 with no
dependant, so whole-project `--wait` failed the instant it finished — even
though everything else was healthy and nrc-init now exits 0.

Smoke now:
  1. `up -d` starts the full stack (one-shots run + deploy as before), then
  2. `up -d --wait <WAIT_SVCS>` waits only for the durable health-checked
     services (openzaak nrc-web acl bff).

Also drops the external `curl localhost:8080/health`: the containerized CI
runner can't reach published host ports at localhost, and each service's
healthcheck already runs inside its container — so `--wait` succeeding IS the
smoke. Documented in docs/runbooks/gitea-actions-gotchas.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 09:03:37 +02:00

108 lines
5.3 KiB
Markdown

# Gitea Actions gotchas
Known differences between Gitea Actions (our CI) and a plain local run, and the
workarounds we adopted. Referenced by `CLAUDE.md` §8.7 and §15.
## Bind mounts don't reach Compose services on the hosted runner
**Symptom.** `make smoke` is green locally but the `compose-smoke` CI job fails
with the OpenZaak init container exiting 1:
```
oz-init-1 | CommandError: Yaml file `/app/setup_configuration/data.yaml` does not exist.
```
Migrations run fine; only the step that reads a **mounted** file fails. The same
class of failure hits any service that bind-mounts a workspace path —
`nrc-init` (its `data.yaml`), `flowable-init` (the BPMN), `keycloak` (the realm
import dir).
**Cause.** The `ubuntu-latest` runner executes the whole job **inside a
container** (`docker.gitea.com/runner-images:ubuntu-latest`). When the job then
runs `docker compose ... up`, Compose talks to the host's Docker daemon and
starts the stack as **sibling containers**. A relative bind mount such as
```yaml
volumes:
- ./openzaak/setup_configuration:/app/setup_configuration:ro
```
is resolved by Compose to an absolute path **inside the job container**
(`/workspace/eho/register-referentie/infra/openzaak/setup_configuration`). The
daemon then looks for that path on **its own host**, doesn't find it, and
auto-creates an **empty directory** to mount. The container starts with an empty
mount point, so the file appears "missing".
This is the classic Docker-in-Docker / sibling-container bind-mount trap. It does
not happen on a runner that executes jobs directly on the host (the previous
self-hosted `respellion-linux` setup), which is why switching to `ubuntu-latest`
exposed it.
**Fix: bake assets into derived images instead of bind-mounting them.** Anything
a Compose service needs at runtime that lives in the repo is `COPY`-ed into a
small derived image, so it is present regardless of where the daemon runs:
| Asset | Derived image | Dockerfile |
|---|---|---|
| OpenZaak `setup_configuration/data.yaml` | `register-referentie/openzaak:dev` | `infra/openzaak/Dockerfile` |
| Open Notificaties `setup_configuration/data.yaml` | `register-referentie/opennotificaties:dev` | `infra/opennotificaties/Dockerfile` |
| Keycloak realm exports | `register-referentie/keycloak:dev` | `infra/keycloak/Dockerfile` |
| `workflows/registratie.bpmn` | `register-referentie/flowable-init:dev` | `build.dockerfile_inline` in the compose files |
The base image tag stays a build `arg` (`OPENZAAK_TAG`, `OPENNOTIFICATIES_TAG`)
so version pinning is unchanged. The three `*-init`/web/celery services that share
a base now share one built image tag, so the bake happens once per `up --build`.
**Why not the alternatives.**
- *Compose `configs:` with inline `content`* — Compose materialises these as a
temp file on the **client** side and bind-mounts it, so it hits the exact same
daemon-can't-see-the-path problem.
- *A self-hosted runner that runs jobs on the host* — works, but reintroduces a
bespoke runner and undoes the move to the hosted `ubuntu-latest` label.
**Consequence for local dev.** There is now no bind mount of these config files,
so the SELinux `:z`/`:Z` relabel flag is no longer needed anywhere in `infra/`,
and rootless Podman no longer needs the files to be world-readable. One mechanism
(build) works on both Podman locally and Docker-in-Docker in CI.
## `--wait` fails on one-shot containers with no dependant
`docker compose up --wait` treats a service that **exits** as a failure of the
"stay up" condition — **unless** another service depends on it with
`condition: service_completed_successfully`. Our init jobs `oz-init` and
`nrc-init` are fine (`openzaak`/`nrc-web` depend on their completion), but
`flowable-init` deploys the BPMN and exits 0 with **no dependant**, so a
whole-project `--wait` fails the moment it exits — even with everything else
healthy. The symptom is a `compose-smoke` failure whose last compose line is:
```
container infra-flowable-init-1 exited (0)
```
**Fix.** The smoke does **not** `--wait` on the whole project. It starts
everything with `up -d`, then `up -d --wait <services>` only for the durable,
health-checked services (`openzaak nrc-web acl bff` — see `WAIT_SVCS` in the
`Makefile`). One-shots still run (and deploy), they just don't gate `--wait`.
This also removed the old external `curl http://localhost:8080/health` check:
the CI job runs in a container and **can't reach published host ports** at
`localhost`, and the per-service healthchecks (which run *inside* the
containers) already prove readiness, so `--wait` succeeding *is* the smoke.
## `--wait` needs an explicit timeout
`docker compose up --wait` defaults to a 60-second timeout in some Compose v2
releases. A cold OpenZaak migrate alone takes ~50 s, so the smoke target passes
`--wait-timeout 300` (see `Makefile`). The 3-minute Definition-of-Done budget
still holds — this just stops `--wait` giving up before the stack is healthy.
## PostGIS readiness vs. `pg_isready`
`pg_isready` reports the server is accepting connections as soon as the TCP port
is open — **before** the `postgis/postgis` image has finished running its
`CREATE EXTENSION postgis` init scripts. An init container that starts migrating
in that window can fail on a missing PostGIS. The db healthchecks therefore add a
`SELECT PostGIS_Version()` probe so dependents wait for the extension, not just
the port.