Files
register-referentie/docs/runbooks/ci.md
T
not 6cfcc4cf83
CI / k8s (push) Successful in 6s
CI / unit (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / mutation (push) Canceled after 0s
CI / verify-stack (push) Canceled after 0s
CI / build (push) Canceled after 9s
CI / lint (push) Canceled after 24s
Deploy to Talos / deploy (push) Successful in 5m1s
ci(deploy): deploy the stack to Talos on merge to main (closes #175) (#176)
## What & why

The chart has been deployable by hand since #25 and linted in CI since #168. This makes a
merged PR actually ship it to the Talos VM on the lab server.

`.gitea/workflows/deploy.yaml` runs on a push to `main` (a squash-merged PR) and on manual
dispatch:

1. **Tunnel** — neither the Kubernetes API nor the in-cluster registry is publicly reachable,
   so 6443, 30500 and 30141 are forwarded over the same SSH hop into the Fedora host that the
   Gitea-runner pipeline uses (`ssh -p 6667 user@labs.respellion.tech`).
2. **Images** — `make k8s-images K8S_REGISTRY=localhost:30500`, pushed *through* the tunnel.
3. **Deploy** — `make k8s-reseed TALOS_HOST=… K8S_REGISTRY=<vm-ip>:30500`, pulled by the node
   from its own NodePort.
4. **Roll** — `rollout restart` + `rollout status` on the nine repo deployments.
5. **Smoke** — `GET /openbaar/register` through the openbaar portal.

Three decisions worth the review:

- **One registry, two names.** The push target (`localhost:30500`, the tunnel) and the pull
  target (`<vm-ip>:30500`, the node's own NodePort) address the same store. The pull name has
  to be the one in the node's registry-mirror patch, which is what makes plain HTTP acceptable.
- **`k8s-reseed`, not `k8s-up`.** A Job's pod template is immutable, so a chart change to any
  bootstrap Job would otherwise fail the upgrade with `cannot patch … with kind Job`. The Jobs
  are idempotent by design, so re-running them every deploy is safe and removes that whole
  class of failure. Cost: a few minutes per deploy, and `seed-zaaktype` needs egress from the VM.
- **No re-run of the checks.** PR CI is the merge gate, so `main` is green by construction.
  Deploys **queue** (`cancel-in-progress: false`) — a `helm upgrade` killed half-way leaves the
  release in `pending-upgrade` and has to be unwedged by hand.

Settings on the repo (already added): secrets `TALOS_SSH_KEY` and `TALOS_KUBECONFIG`
(base64, and its `server:` must be `https://127.0.0.1:6443` — Talos puts `127.0.0.1` in the
apiserver cert SANs, so TLS still verifies through the tunnel); variables `TALOS_VM_IP`
(default `192.168.122.173`) and `TALOS_HOST` (default `localhost`).

Closes #175

## Definition of Done

- [x] Linked Gitea issue (above).
- [ ] Failing test committed before the implementation — **n/a**: this is a deployment
      workflow with no unit under test. Its check is the run itself: `rollout status` and the
      public-register smoke both have to pass or the job fails. `make k8s-lint` / `make k8s-drift`
      (#168) already gate the chart it deploys.
- [x] Implementation — one workflow file, no production code touched.
- [x] Conventional Commits referencing the issue (`refs #175`).
- [ ] CI green — awaiting the run on this PR.
- [x] `docker compose up` unaffected — no service, image or compose file is touched.
- [x] Docs updated — `docs/runbooks/kubernetes-talos.md` §9 (the tunnel, the two registry
      names, the secrets table, the smoke) and a pointer from `docs/runbooks/ci.md`.
- [x] No ADR needed: no new dependency (kubectl/helm/crane are already prerequisites of the
      `k8s-*` targets), no service boundary moved, no CLAUDE.md §8 rule bent.
- [ ] Demo note — not user-visible.

## Notes for reviewers

- **The first deploy is the real test.** It cannot be dry-run: the tunnel, the secrets and the
  registry only exist on the lab server. Merging is how we find out; `Pods on failure` dumps
  `get pods,jobs` if it doesn't.
- **Known gap — the portals still aren't browsable.** PKCE needs a secure context, so a
  NodePort on an IP can't serve them (runbook §5); they need `make k8s-portals` or an SSH
  forward. Giving the server a hostname + TLS is the follow-up, and is where `TALOS_HOST`
  stops defaulting to `localhost`.
- **Databases are `emptyDir`.** Any change to a database pod's template wipes it; the
  `k8s-reseed` in the deploy re-runs the bootstrap, so the stack recovers, but submitted
  registrations do not. Persistence is runbook §6.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #176
2026-09-18 13:53:58 +00:00

8.2 KiB

CI runbook — Gitea Actions

Status: active. The workflow .gitea/workflows/ci.yaml runs on Gitea's hosted ubuntu-latest runner — no self-hosted runner required. make ci is still the local gate — it runs the same checks via the same make targets, with one exception: the k8s job's targets are not in make ci, because helm is optional for everyone not deploying to Kubernetes. Run make k8s-lint k8s-drift by hand after touching the chart or the compose file.

The pipeline

.gitea/workflows/ci.yaml runs on every push and pull request to main. Each job calls a make target — the single source of truth for the checks, so local and CI cannot drift:

Job Target Needs
lint make lintdotnet format … --verify-no-changes .NET 10 SDK
build make builddotnet build … -c Release .NET 10 SDK
unit make unitdotnet test … -c Release --filter "Category!=Integration" .NET 10 SDK
frontend make frontend → Nx lint/test/build for the four portals pnpm + Node
k8s make k8s-lint (render + schema-check the Helm chart) → make k8s-drift (chart still describes the same stack as infra/docker-compose.yml) pinned helm binary + docker compose
mutation make mutationdotnet tool restoredotnet stryker (ACL); uploads the HTML report as an artifact .NET 10 SDK
verify-stack the single live-stack stage — steps: make verify-up (full stack up + health, the DoD smoke) → make verify-acl (ACL ↔ OpenZaak) → make verify-nrc (OpenZaak → NRC delivery) → make down container engine + egress (base images, nuget, selectielijst.openzaak.nl)

Why one verify-stack job, not three. The single self-hosted runner runs jobs sequentially, so booting OpenZaak once (instead of once per check) is the cheapest layout (issue #58). It subsumes the old integration, notifications, and compose-smoke jobs — the bring-up step is the "compose up reaches green health" gate. No setup-dotnet: the ACL test runs in a built image and every check reaches services by container IP (the runner can't reach published ports — see gitea-actions-gotchas.md §5/§6).

A second workflow, .gitea/workflows/deploy.yaml, deploys the stack to the Talos cluster on the lab server when a PR is merged to main — see kubernetes-talos.md §9 for its secrets and the SSH tunnel it needs.

All uses: references are absolute, tag-pinned URLs (https://github.com/actions/checkout@v4, https://github.com/actions/setup-dotnet@v4) per CLAUDE.md §8.7 and §15 — Gitea Actions resolves them from GitHub.

verify-stack runs on a containerized runner. Workspace bind mounts do not reach the sibling containers Compose starts, so config/assets are streamed into external named volumes via docker cp (infra/seed-config.sh), and the upstream images are used verbatim (no build). If you add a service that needs a repo file at runtime, seed it the same way — don't bind-mount it. Note: bare docker compose up no longer self-seeds; use make up. See gitea-actions-gotchas.md.

Mutation testing (the ratchet)

The mutation job enforces test strength, not just coverage (CLAUDE.md §5). Stryker.NET is pinned as a local dotnet tool (.config/dotnet-tools.json), so it runs identically locally and in CI:

make mutation     # dotnet tool restore + dotnet stryker on the ACL

Config lives in services/acl/stryker-config.json. It runs in solution mode against Acl.slnx, mutating the two projects under test (Acl.Application, Acl.Infrastructure); Acl.Api has no tests and is skipped.

Baseline (the ratchet): the ACL is the first service with branching logic, so it sets the repo-wide baseline. Observed score 95%; enforced break threshold 90% (one-mutant headroom over the ~20-mutant surface). Stryker exits non-zero — failing the job — when the score drops below break. Per §5 the baseline only moves up, and only as a slice's stated outcome; never lower it. New services add their own mutation run as they gain logic.

The HTML report is written to services/acl/StrykerOutput/<timestamp>/reports/ (git-ignored); open it to see survived vs. killed mutants.

In CI the mutation job publishes that report as the acl-mutation-report artifact (download it from the run's summary page). The upload step uses if: always(), so the report is available even when the ratchet fails — which is exactly when you want to inspect the survivors. It is the repo's first use of actions/upload-artifact, pinned to @v3: @v4 refuses to run on Gitea (its @actions/artifact v2 library blocks any non-github.com server as "GHES"), while @v3 speaks the artifact protocol Gitea implements. See gitea-actions-gotchas.md §4 (§15).

Running the stack locally without make (Windows / Docker Desktop)

make and the bash helpers assume a Unix shell. To bring the whole stack up on a machine without them (e.g. Windows + Docker Desktop), use the local compose file, which bind-mounts the config instead of seeding volumes — so it needs no make, no seed step, and no bash:

docker compose -f infra/docker-compose.local.yml up -d --build          # any engine
docker compose -f infra/docker-compose.local.yml up -d --build --wait    # Docker Desktop (Compose v2)
docker compose -f infra/docker-compose.local.yml down --volumes

On Linux/macOS the same thing is wrapped as make local / make local-down.

infra/docker-compose.local.yml mirrors the canonical infra/docker-compose.yml but swaps the external config volumes for bind mounts — valid locally because a local daemon can see the working directory (the seed/volume dance only exists for the containerized CI runner). Keep the two files in sync.

Running CI locally (make ci)

make ci runs the exact same checks as the pipeline — handy to run before pushing:

make ci          # lint + build + unit + mutation + verify — mirrors the pipeline
make lint        # or a single stage
make mutation    # Stryker.NET ratchet on the ACL
make verify      # the live-stack stage: full stack up once → ACL + NRC checks → down

make verify mirrors the CI verify-stack job: it boots the full stack once and runs both the ACL ↔ OpenZaak and OpenZaak → NRC checks against it. For fast, single-concern local iteration use a lighter throwaway stack instead:

make integration          # ACL ↔ OpenZaak only (no NRC)
make verify-notifications # OpenZaak → NRC delivery only

Prerequisites: .NET 10 SDK, a container engine with Compose v2, and curl.

On a rootless Podman box (the default dev setup here), the smoke target needs the Podman API socket and a Compose provider:

systemctl --user enable --now podman.socket            # start the API socket
ln -sf "$(command -v podman)" ~/.local/bin/docker      # docker -> podman shim
# install Docker Compose v2 into ~/.local/bin as `docker-compose` (the provider)

The Makefile auto-points DOCKER_HOST at /run/user/$(id -u)/podman/podman.sock when that socket exists and DOCKER_HOST is unset, so make smoke "just works" locally while leaving real Docker hosts / CI runners untouched.

Runner: ubuntu-latest

All jobs run on Gitea's hosted ubuntu-latest runner — no self-hosted runner setup is required. The hosted runner ships with Docker and Docker Compose v2, so make smoke (docker compose … up --wait) works without extra configuration.

If Gitea's hosted runners are unavailable and a self-hosted fallback is needed, register an act_runner with the ubuntu-latest label:

VER=0.2.11
curl -fsSL -o /usr/local/bin/act_runner \
  "https://dl.gitea.com/act_runner/${VER}/act_runner-${VER}-linux-amd64"
chmod +x /usr/local/bin/act_runner

act_runner register --no-interactive \
  --instance https://git.labs.respellion.tech \
  --token <REGISTRATION_TOKEN> \
  --name respellion-ci-1 \
  --labels "ubuntu-latest:docker://node:20-bookworm"

act_runner daemon