Files
register-referentie/docs/runbooks/ci.md
T
not 6cfcc4cf83
CI / k8s (push) Successful in 6s
CI / unit (push) Canceled after 0s
CI / frontend (push) Canceled after 0s
CI / mutation (push) Canceled after 0s
CI / verify-stack (push) Canceled after 0s
CI / build (push) Canceled after 9s
CI / lint (push) Canceled after 24s
Deploy to Talos / deploy (push) Successful in 5m1s
ci(deploy): deploy the stack to Talos on merge to main (closes #175) (#176)
## What & why

The chart has been deployable by hand since #25 and linted in CI since #168. This makes a
merged PR actually ship it to the Talos VM on the lab server.

`.gitea/workflows/deploy.yaml` runs on a push to `main` (a squash-merged PR) and on manual
dispatch:

1. **Tunnel** — neither the Kubernetes API nor the in-cluster registry is publicly reachable,
   so 6443, 30500 and 30141 are forwarded over the same SSH hop into the Fedora host that the
   Gitea-runner pipeline uses (`ssh -p 6667 user@labs.respellion.tech`).
2. **Images** — `make k8s-images K8S_REGISTRY=localhost:30500`, pushed *through* the tunnel.
3. **Deploy** — `make k8s-reseed TALOS_HOST=… K8S_REGISTRY=<vm-ip>:30500`, pulled by the node
   from its own NodePort.
4. **Roll** — `rollout restart` + `rollout status` on the nine repo deployments.
5. **Smoke** — `GET /openbaar/register` through the openbaar portal.

Three decisions worth the review:

- **One registry, two names.** The push target (`localhost:30500`, the tunnel) and the pull
  target (`<vm-ip>:30500`, the node's own NodePort) address the same store. The pull name has
  to be the one in the node's registry-mirror patch, which is what makes plain HTTP acceptable.
- **`k8s-reseed`, not `k8s-up`.** A Job's pod template is immutable, so a chart change to any
  bootstrap Job would otherwise fail the upgrade with `cannot patch … with kind Job`. The Jobs
  are idempotent by design, so re-running them every deploy is safe and removes that whole
  class of failure. Cost: a few minutes per deploy, and `seed-zaaktype` needs egress from the VM.
- **No re-run of the checks.** PR CI is the merge gate, so `main` is green by construction.
  Deploys **queue** (`cancel-in-progress: false`) — a `helm upgrade` killed half-way leaves the
  release in `pending-upgrade` and has to be unwedged by hand.

Settings on the repo (already added): secrets `TALOS_SSH_KEY` and `TALOS_KUBECONFIG`
(base64, and its `server:` must be `https://127.0.0.1:6443` — Talos puts `127.0.0.1` in the
apiserver cert SANs, so TLS still verifies through the tunnel); variables `TALOS_VM_IP`
(default `192.168.122.173`) and `TALOS_HOST` (default `localhost`).

Closes #175

## Definition of Done

- [x] Linked Gitea issue (above).
- [ ] Failing test committed before the implementation — **n/a**: this is a deployment
      workflow with no unit under test. Its check is the run itself: `rollout status` and the
      public-register smoke both have to pass or the job fails. `make k8s-lint` / `make k8s-drift`
      (#168) already gate the chart it deploys.
- [x] Implementation — one workflow file, no production code touched.
- [x] Conventional Commits referencing the issue (`refs #175`).
- [ ] CI green — awaiting the run on this PR.
- [x] `docker compose up` unaffected — no service, image or compose file is touched.
- [x] Docs updated — `docs/runbooks/kubernetes-talos.md` §9 (the tunnel, the two registry
      names, the secrets table, the smoke) and a pointer from `docs/runbooks/ci.md`.
- [x] No ADR needed: no new dependency (kubectl/helm/crane are already prerequisites of the
      `k8s-*` targets), no service boundary moved, no CLAUDE.md §8 rule bent.
- [ ] Demo note — not user-visible.

## Notes for reviewers

- **The first deploy is the real test.** It cannot be dry-run: the tunnel, the secrets and the
  registry only exist on the lab server. Merging is how we find out; `Pods on failure` dumps
  `get pods,jobs` if it doesn't.
- **Known gap — the portals still aren't browsable.** PKCE needs a secure context, so a
  NodePort on an IP can't serve them (runbook §5); they need `make k8s-portals` or an SSH
  forward. Giving the server a hostname + TLS is the follow-up, and is where `TALOS_HOST`
  stops defaulting to `localhost`.
- **Databases are `emptyDir`.** Any change to a database pod's template wipes it; the
  `k8s-reseed` in the deploy re-runs the bootstrap, so the stack recovers, but submitted
  registrations do not. Persistence is runbook §6.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #176
2026-09-18 13:53:58 +00:00

160 lines
8.2 KiB
Markdown

# CI runbook — Gitea Actions
> **Status: active.** The workflow `.gitea/workflows/ci.yaml` runs on Gitea's
> hosted `ubuntu-latest` runner — no self-hosted runner required.
> **`make ci` is still the local gate** — it runs the same checks via the same
> `make` targets, with one exception: the `k8s` job's targets are not in `make ci`,
> because `helm` is optional for everyone not deploying to Kubernetes. Run
> `make k8s-lint k8s-drift` by hand after touching the chart or the compose file.
## The pipeline
`.gitea/workflows/ci.yaml` runs on every push and pull request to `main`. Each job
calls a `make` target — the **single source of truth** for the checks, so local
and CI cannot drift:
| Job | Target | Needs |
|---|---|---|
| `lint` | `make lint``dotnet format … --verify-no-changes` | .NET 10 SDK |
| `build` | `make build``dotnet build … -c Release` | .NET 10 SDK |
| `unit` | `make unit``dotnet test … -c Release --filter "Category!=Integration"` | .NET 10 SDK |
| `frontend` | `make frontend` → Nx lint/test/build for the four portals | pnpm + Node |
| `k8s` | `make k8s-lint` (render + schema-check the Helm chart) → `make k8s-drift` (chart still describes the same stack as `infra/docker-compose.yml`) | pinned `helm` binary + `docker compose` |
| `mutation` | `make mutation``dotnet tool restore``dotnet stryker` (ACL); uploads the HTML report as an artifact | .NET 10 SDK |
| `verify-stack` | the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
> **Why one `verify-stack` job, not three.** The single self-hosted runner runs jobs
> **sequentially**, so booting OpenZaak once (instead of once per check) is the
> cheapest layout (issue #58). It subsumes the old `integration`, `notifications`, and
> `compose-smoke` jobs — the bring-up step *is* the "compose up reaches green health"
> gate. No `setup-dotnet`: the ACL test runs in a built image and every check reaches
> services by **container IP** (the runner can't reach published ports — see
> [gitea-actions-gotchas.md §5/§6](gitea-actions-gotchas.md)).
A second workflow, `.gitea/workflows/deploy.yaml`, deploys the stack to the Talos
cluster on the lab server when a PR is merged to `main` — see
[kubernetes-talos.md §9](kubernetes-talos.md) for its secrets and the SSH tunnel it needs.
All `uses:` references are absolute, tag-pinned URLs (`https://github.com/actions/checkout@v4`,
`https://github.com/actions/setup-dotnet@v4`) per CLAUDE.md §8.7 and §15 — Gitea
Actions resolves them from GitHub.
> **`verify-stack` runs on a containerized runner.** Workspace bind mounts do
> **not** reach the sibling containers Compose starts, so config/assets are
> streamed into external named volumes via `docker cp` (`infra/seed-config.sh`),
> and the upstream images are used verbatim (no build). If you add a service that
> needs a repo file at runtime, seed it the same way — don't bind-mount it. Note:
> bare `docker compose up` no longer self-seeds; use `make up`. See
> [gitea-actions-gotchas.md](gitea-actions-gotchas.md).
## Mutation testing (the ratchet)
The `mutation` job enforces test *strength*, not just coverage (CLAUDE.md §5).
[Stryker.NET](https://stryker-mutator.io/docs/stryker-net/) is pinned as a local
dotnet tool (`.config/dotnet-tools.json`), so it runs identically locally and in CI:
```bash
make mutation # dotnet tool restore + dotnet stryker on the ACL
```
Config lives in [`services/acl/stryker-config.json`](../../services/acl/stryker-config.json).
It runs in **solution mode** against `Acl.slnx`, mutating the two projects under test
(`Acl.Application`, `Acl.Infrastructure`); `Acl.Api` has no tests and is skipped.
**Baseline (the ratchet):** the ACL is the first service with branching logic, so it
sets the repo-wide baseline. Observed score **95%**; enforced `break` threshold **90%**
(one-mutant headroom over the ~20-mutant surface). Stryker exits non-zero — failing the
job — when the score drops below `break`. Per §5 the baseline only moves **up**, and only
as a slice's stated outcome; never lower it. New services add their own mutation run as
they gain logic.
The HTML report is written to `services/acl/StrykerOutput/<timestamp>/reports/` (git-ignored);
open it to see survived vs. killed mutants.
In CI the `mutation` job publishes that report as the **`acl-mutation-report`** artifact
(download it from the run's summary page). The upload step uses `if: always()`, so the
report is available even when the ratchet *fails* — which is exactly when you want to inspect
the survivors. It is the repo's first use of `actions/upload-artifact`, pinned to **`@v3`**:
`@v4` refuses to run on Gitea (its `@actions/artifact` v2 library blocks any non-github.com
server as "GHES"), while `@v3` speaks the artifact protocol Gitea implements. See
[gitea-actions-gotchas.md §4](gitea-actions-gotchas.md) (§15).
## Running the stack locally without `make` (Windows / Docker Desktop)
`make` and the bash helpers assume a Unix shell. To bring the whole stack up on a
machine without them (e.g. Windows + Docker Desktop), use the **local compose
file**, which bind-mounts the config instead of seeding volumes — so it needs no
`make`, no seed step, and no bash:
```bash
docker compose -f infra/docker-compose.local.yml up -d --build # any engine
docker compose -f infra/docker-compose.local.yml up -d --build --wait # Docker Desktop (Compose v2)
docker compose -f infra/docker-compose.local.yml down --volumes
```
On Linux/macOS the same thing is wrapped as `make local` / `make local-down`.
`infra/docker-compose.local.yml` mirrors the canonical `infra/docker-compose.yml`
but swaps the external config volumes for bind mounts — valid locally because a
local daemon can see the working directory (the seed/volume dance only exists for
the containerized CI runner). Keep the two files in sync.
## Running CI locally (`make ci`)
`make ci` runs the exact same checks as the pipeline — handy to run before pushing:
```bash
make ci # lint + build + unit + mutation + verify — mirrors the pipeline
make lint # or a single stage
make mutation # Stryker.NET ratchet on the ACL
make verify # the live-stack stage: full stack up once → ACL + NRC checks → down
```
> **`make verify`** mirrors the CI `verify-stack` job: it boots the full stack once and
> runs both the ACL ↔ OpenZaak and OpenZaak → NRC checks against it. For fast,
> single-concern local iteration use a lighter throwaway stack instead:
>
> ```bash
> make integration # ACL ↔ OpenZaak only (no NRC)
> make verify-notifications # OpenZaak → NRC delivery only
> ```
**Prerequisites:** .NET 10 SDK, a container engine with Compose v2, and `curl`.
On a **rootless Podman** box (the default dev setup here), the `smoke` target needs
the Podman API socket and a Compose provider:
```bash
systemctl --user enable --now podman.socket # start the API socket
ln -sf "$(command -v podman)" ~/.local/bin/docker # docker -> podman shim
# install Docker Compose v2 into ~/.local/bin as `docker-compose` (the provider)
```
The Makefile auto-points `DOCKER_HOST` at `/run/user/$(id -u)/podman/podman.sock`
when that socket exists and `DOCKER_HOST` is unset, so `make smoke` "just works"
locally while leaving real Docker hosts / CI runners untouched.
## Runner: `ubuntu-latest`
All jobs run on Gitea's hosted **`ubuntu-latest`** runner — no self-hosted runner
setup is required. The hosted runner ships with Docker and Docker Compose v2, so
`make smoke` (`docker compose … up --wait`) works without extra configuration.
If Gitea's hosted runners are unavailable and a self-hosted fallback is needed,
register an `act_runner` with the `ubuntu-latest` label:
```bash
VER=0.2.11
curl -fsSL -o /usr/local/bin/act_runner \
"https://dl.gitea.com/act_runner/${VER}/act_runner-${VER}-linux-amd64"
chmod +x /usr/local/bin/act_runner
act_runner register --no-interactive \
--instance https://git.labs.respellion.tech \
--token <REGISTRATION_TOKEN> \
--name respellion-ci-1 \
--labels "ubuntu-latest:docker://node:20-bookworm"
act_runner daemon
```