diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index d3b6e36..5acf277 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -63,11 +63,22 @@ jobs: path: services/acl/StrykerOutput/**/reports/mutation-report.html if-no-files-found: warn - # NOTE: there is deliberately no `integration` job here yet. `make integration` - # runs the ACL ↔ real-OpenZaak test locally / on a host-executing runner, but on - # the hosted runner a process on the runner cannot reach the stack's published - # ports (sibling containers — see gitea-actions-gotchas.md §5). Running it inside - # the compose network is tracked in #55. + integration: + runs-on: ubuntu-latest + steps: + - uses: https://github.com/actions/checkout@v4 + # No setup-dotnet: `make integration` runs the seed and the test as containers + # *inside* the OpenZaak compose network (reaching it by container IP), so dotnet + # lives in the test image and the runner needs only Docker. This sidesteps the + # runner being unable to reach published ports (gitea-actions-gotchas.md §5). + # Needs egress to pull base images + nuget + selectielijst.openzaak.nl. + - run: make integration + - name: dump OpenZaak logs on failure + if: failure() + run: docker compose -f infra/openzaak/docker-compose.yml logs --no-color --tail=80 oz-init openzaak 2>&1 || true + - name: tear down on failure + if: failure() + run: docker compose -f infra/openzaak/docker-compose.yml down --volumes 2>&1 || true compose-smoke: runs-on: ubuntu-latest diff --git a/Makefile b/Makefile index 21ee91b..8fde334 100644 --- a/Makefile +++ b/Makefile @@ -106,20 +106,14 @@ local-down: changelog: git-cliff --output CHANGELOG.md -## integration: ACL integration tests against a real OpenZaak (S-04a, #46). Brings -## the stack up, seeds a PUBLISHED BIG zaaktype (OZ_PUBLISH=1, so OpenZaak accepts a -## real zaak POST), runs the Integration-category tests, then always tears down. -## Kept out of `unit`/`mutation` because it needs the live stack. See ADR-0006. -integration: openzaak-up - @bash -c 'set -e; \ - for i in $$(seq 1 60); do \ - c=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/catalogi/api/v1/ || true); \ - [ "$$c" = "200" ] && break; sleep 3; done; echo "OpenZaak ready ($$c)"; \ - OZ_PUBLISH=1 python3 infra/openzaak/seed_catalogus.py; \ - rc=0; dotnet test $(SLN) -c Release --filter "Category=Integration" || rc=$$?; \ - docker compose -f $(OZ_COMPOSE) down --volumes >/dev/null 2>&1 || true; \ - docker volume rm -f rr-oz-config >/dev/null 2>&1 || true; \ - exit $$rc' +## integration: ACL integration tests against a real OpenZaak (S-04a, #46). The +## seed and the test run inside the compose network (reaching http://openzaak:8000), +## so this works on the hosted CI runner where a runner process can't reach the +## stack's published ports. Brings the stack up, seeds a PUBLISHED BIG zaaktype, +## runs the Integration-category tests, then always tears down. Kept out of +## `unit`/`mutation` because it needs the live stack. See infra/run-integration.sh + ADR-0006. +integration: + bash infra/run-integration.sh ## openzaak-up: start the OpenZaak stack (migrations run on first start) openzaak-up: diff --git a/docs/architecture/adr-0006-integration-test-provisioning.md b/docs/architecture/adr-0006-integration-test-provisioning.md index 5494736..2fd8e00 100644 --- a/docs/architecture/adr-0006-integration-test-provisioning.md +++ b/docs/architecture/adr-0006-integration-test-provisioning.md @@ -47,8 +47,9 @@ itself. No new test dependency is added.** - **The lane is kept out of the fast checks.** `make unit` runs with `--filter "Category!=Integration"`; Stryker is pinned to `Acl.Tests` (`test-projects`), so neither the unit nor the mutation lane needs a live stack. A new `make integration` target - brings the stack up, seeds, runs the lane, and always tears down. This matches `make` being - the single source of truth (ADR-0005). + (`infra/run-integration.sh`) brings the stack up, seeds, runs the lane, and always tears down + — mirrored by a Gitea Actions `integration` job. This matches `make` being the single source + of truth (ADR-0005). - **Publishing is opt-in in the seed.** `infra/openzaak/seed_catalogus.py` gains an `OZ_PUBLISH=1` path that adds the relations OpenZaak's publish requires — two statustypen (begin/eind), a roltype, and a resultaattype whose Selectielijst procestype is matched onto @@ -70,12 +71,13 @@ itself. No new test dependency is added.** in production) but it is a network touchpoint, and a CI environment without egress would need a local Selectielijst service or a recorded fixture. `OZ_SELECTIELIJST` overrides the base URL. - **Cost:** the lane needs the stack up first, so it is separate from the fast lanes. -- **Not yet a hosted-runner job.** `make integration` passes locally and on a host-executing - runner, but on Gitea's hosted runner a process *on the runner* cannot reach the stack's - published ports — Compose starts sibling containers via the host daemon, so the ports bind to - the host, not the runner (gitea-actions-gotchas.md §5, the same split as §1). Running the test - and seed *inside* the compose network (via a built image, not bind mounts) to gate it in CI is - tracked in **#55**. Until then the lane runs locally and is not a merge gate. +- **Runs on the hosted runner.** A process *on* the runner can't reach the stack's published + ports (Compose starts sibling containers via the host daemon — gitea-actions-gotchas.md §5, + same split as §1), so `infra/run-integration.sh` runs both the seed and the test as containers + *joined to the OpenZaak network*, reaching it by **container IP** (a single-label host like + `openzaak` isn't URL-valid for OpenZaak's own `URLValidator`; an IPv4 literal is). Code is + delivered by image build / `docker cp`, never bind mounts. The CI job therefore needs only + Docker — no `setup-dotnet`. (This closed the follow-up that was originally split out as #55.) ## Alternatives considered diff --git a/docs/runbooks/ci.md b/docs/runbooks/ci.md index d925115..abd035e 100644 --- a/docs/runbooks/ci.md +++ b/docs/runbooks/ci.md @@ -17,13 +17,13 @@ and CI cannot drift: | `build` | `make build` → `dotnet build … -c Release` | .NET 10 SDK | | `unit` | `make unit` → `dotnet test … -c Release --filter "Category!=Integration"` | .NET 10 SDK | | `mutation` | `make mutation` → `dotnet tool restore` → `dotnet stryker` (ACL); uploads the HTML report as an artifact | .NET 10 SDK | +| `integration` | `make integration` → `infra/run-integration.sh`: OpenZaak up → seed a **published** BIG zaaktype + run `Acl.IntegrationTests` **as containers inside the compose network** → tear down | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) | | `compose-smoke` | `make smoke` → seed config volumes → `up -d` (full stack) → `up --wait` durable services → `down` | container engine + compose v2 | -> **`make integration` is not a hosted-runner job yet.** The ACL ↔ real-OpenZaak -> test (`make integration`, ADR-0006) passes locally and on a host-executing runner, -> but a process on the hosted runner can't reach the stack's published ports -> (sibling containers — see [gitea-actions-gotchas.md §5](gitea-actions-gotchas.md)). -> Run it inside the compose network to gate it in CI — tracked in **#55**. +> **The `integration` job needs no `setup-dotnet`.** dotnet runs inside the test +> image, and both the seed and the test join the OpenZaak network and reach it by +> container IP — so the runner never has to reach a published port +> (see [gitea-actions-gotchas.md §5](gitea-actions-gotchas.md)). All `uses:` references are absolute, tag-pinned URLs (`https://github.com/actions/checkout@v4`, `https://github.com/actions/setup-dotnet@v4`) per CLAUDE.md §8.7 and §15 — Gitea diff --git a/docs/runbooks/gitea-actions-gotchas.md b/docs/runbooks/gitea-actions-gotchas.md index 1967454..c9c9ae4 100644 --- a/docs/runbooks/gitea-actions-gotchas.md +++ b/docs/runbooks/gitea-actions-gotchas.md @@ -152,6 +152,8 @@ service by name (`http://openzaak:8000`). For a test/seed that needs the repo's code, deliver it via a **built image** (not a bind mount — §1), then `docker run --network _cg …`. -**Status** — `make integration` (the ACL ↔ real-OpenZaak test, ADR-0006) therefore -runs locally / on a host-executing runner only. Gating it on the hosted runner via -the compose network is tracked in **#55**. +**Applied** — `make integration` (the ACL ↔ real-OpenZaak test, ADR-0006) does +exactly this: `infra/run-integration.sh` runs the seed and the test as containers on +the OpenZaak network and reaches it by **container IP** (a single-label service name +like `openzaak` isn't URL-valid — OpenZaak echoes the request host into the URLs it +returns and then rejects them with Django's `URLValidator`; an IPv4 literal passes). diff --git a/infra/run-integration.sh b/infra/run-integration.sh new file mode 100755 index 0000000..c43fb25 --- /dev/null +++ b/infra/run-integration.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# +# Run the ACL ↔ real-OpenZaak integration test (S-04a / #46) end to end. +# +# Everything that talks to OpenZaak runs *inside* the compose network and reaches +# it by service name (http://openzaak:8000) — the hosted CI runner can't reach the +# stack's published ports (sibling containers) and bind mounts don't reach the +# daemon either (gitea-actions-gotchas.md §1/§5). So we use only plain docker +# primitives (run / create / cp / build) — portable across docker compose (CI) and +# podman-compose (local), exactly like infra/seed-config.sh. See ADR-0006. +# +# Steps: bring OpenZaak up → wait for it healthy → seed a PUBLISHED BIG zaaktype +# (a seed container on the network) → build + run the test container on the network +# → always tear down. +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +root="$(cd "$here/.." && pwd)" +OZ_COMPOSE="$here/openzaak/docker-compose.yml" + +cleanup() { + docker compose -f "$OZ_COMPOSE" down --volumes >/dev/null 2>&1 || true + docker volume rm -f rr-oz-config >/dev/null 2>&1 || true +} +trap cleanup EXIT + +echo ">> bringing OpenZaak up" +bash "$here/seed-config.sh" oz +docker compose -f "$OZ_COMPOSE" up -d + +echo ">> waiting for the OpenZaak API container to be healthy" +# Match the API container under both docker compose (openzaak-openzaak-1) and +# podman-compose (openzaak_openzaak_1) naming; the regex excludes oz-db / oz-redis. +api="" +for _ in $(seq 1 140); do + api="$(docker ps -q --filter 'name=openzaak[-_]openzaak' | head -1)" + if [ -n "$api" ]; then + status="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{end}}' "$api" 2>/dev/null || true)" + [ "$status" = "healthy" ] && break + fi + sleep 3 +done +[ -n "$api" ] || { echo "ERROR: OpenZaak API container never appeared" >&2; exit 1; } +[ "${status:-}" = "healthy" ] || { echo "ERROR: OpenZaak not healthy (status=${status:-none})" >&2; exit 1; } + +# The network the API container is attached to — joined by the seed + test below. +net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$api" | head -1)" +# Reach OpenZaak by container IP, not by the service name. OpenZaak echoes its +# request Host into the self-referential URLs it returns, then validates those URLs +# with Django's URLValidator — which rejects a single-label host like `openzaak` +# ("Voer een geldige URL in") while accepting an IPv4 literal (and `localhost`). +oz_ip="$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$api")" +oz_base="http://${oz_ip}:8000" +echo ">> OpenZaak healthy on network $net at $oz_base" + +echo ">> seeding a published BIG zaaktype (OZ_PUBLISH=1, inside the network)" +sid="$(docker create --network "$net" \ + -e "OZ_BASE=$oz_base" -e OZ_PUBLISH=1 \ + python:3-slim python /seed_catalogus.py)" +docker cp "$here/openzaak/seed_catalogus.py" "$sid:/seed_catalogus.py" +docker start -a "$sid" +docker rm -f "$sid" >/dev/null + +echo ">> building the integration test image" +docker build -f "$root/services/acl/Dockerfile.integration" -t rr-acl-integration "$root/services/acl" + +echo ">> running the integration tests (inside the network)" +docker run --rm --network "$net" -e "OZ_BASE=$oz_base" rr-acl-integration diff --git a/services/acl/Dockerfile.integration b/services/acl/Dockerfile.integration new file mode 100644 index 0000000..969843d --- /dev/null +++ b/services/acl/Dockerfile.integration @@ -0,0 +1,26 @@ +# Runs the ACL integration tests (Category=Integration) from *inside* the compose +# network, so they reach OpenZaak at http://openzaak:8000 by service name. On the +# hosted CI runner a process on the runner can't reach the stack's published ports +# (sibling containers — gitea-actions-gotchas.md §5), so the test runs as a +# container joined to that network instead. See ADR-0006 / #55. +# +# Build context is services/acl (like the service Dockerfile). dotnet lives in this +# image, so the CI `integration` job needs only Docker — no setup-dotnet step. +FROM mcr.microsoft.com/dotnet/sdk:10.0 +WORKDIR /src + +# Restore first (cached unless the .csproj files change). The integration test +# project pulls in Acl.Application + Acl.Infrastructure via its ProjectReferences. +COPY Acl.Application/Acl.Application.csproj Acl.Application/ +COPY Acl.Infrastructure/Acl.Infrastructure.csproj Acl.Infrastructure/ +COPY Acl.IntegrationTests/Acl.IntegrationTests.csproj Acl.IntegrationTests/ +RUN dotnet restore Acl.IntegrationTests/Acl.IntegrationTests.csproj + +COPY Acl.Application/ Acl.Application/ +COPY Acl.Infrastructure/ Acl.Infrastructure/ +COPY Acl.IntegrationTests/ Acl.IntegrationTests/ + +# OZ_BASE is supplied at run time (the OpenZaak container IP — see run-integration.sh, +# which passes `-e OZ_BASE=http://:8000`; a single-label host is not URL-valid). +ENTRYPOINT ["dotnet", "test", "Acl.IntegrationTests/Acl.IntegrationTests.csproj", \ + "-c", "Release", "--filter", "Category=Integration"]