Files
register-referentie/.gitea/workflows/ci.yaml
T
not d5dfbdc0b2
CI / build (push) Successful in 1m47s
CI / lint (push) Successful in 1m59s
CI / unit (push) Successful in 1m57s
CI / frontend (push) Successful in 3m58s
CI / mutation (push) Successful in 6m57s
CI / verify-stack (push) Successful in 8m29s
feat(obs): Prometheus metrics on /metrics + golden-signal Grafana dashboard (closes #124) (#129)
## What & why

S-16c, the last of the S-16 (#17) split, on top of the backplane (#122) and distributed tracing (#123). The five .NET services now expose OpenTelemetry **metrics** in Prometheus format at `/metrics`; Prometheus scrapes each (one job per service); and Grafana ships a pre-built **Request path — golden signals** dashboard (traffic / errors / latency / saturation), split by service.

Closes #124

### How

- Each service adds `.WithMetrics(AddAspNetCoreInstrumentation + AddHttpClientInstrumentation + AddMeter("System.Runtime") + AddPrometheusExporter)` and maps `/metrics`. Same shape as the S-16b tracing wiring already in these `Program.cs` files.
- `infra/observability/prometheus/prometheus.yml`: one scrape job per service (`acl`, `domain`, `bff`, `event-subscriber`, `projection-api`), reached by compose service name.
- `infra/observability/grafana/provisioning/dashboards/`: dashboard provider + `golden-signals.json` (baked into the Grafana image by the existing `COPY provisioning/`).
- `verify-metrics` (new CI verify-stack step + Makefile target): generates BFF traffic and asserts Prometheus scraped the golden-signal metric from every service. Mirrors `verify-tracing`.

### Dependency (CLAUDE.md §13/§14)

Adds `OpenTelemetry.Exporter.Prometheus.AspNetCore` `1.17.0-beta.1` (matched to the `1.17.0` core already in use). It gives the OTel-native `/metrics` pull endpoint; replacing it would mean hand-rolling Prometheus exposition over a `MeterListener`; the risk is that it is a **prerelease** package (the whole OTel .NET Prometheus line is `-beta`) — pinned, wired only in `Program.cs`, and gated by `verify-metrics`. Recorded in **ADR-0024**.

## Definition of Done

- [x] Linked Gitea issue (#124).
- [x] Failing test committed before the implementation (`test(bff): /metrics exposes http-server request duration`).
- [x] Implementation makes the test pass.
- [ ] CI green — pending Gitea Actions run.
- [x] `docker compose up` reaches green health within 3 min (backplane images unchanged in shape; not on the health gate, ADR-0023).
- [x] Docs updated — demo-script S-16c entry.
- [x] ADR added — ADR-0024.
- [x] Demo note in `docs/demo-script.md`.

## Notes for reviewers

- `/health` polls are counted as traffic (metrics aren't path-filtered, unlike traces). Fine for a demo dashboard and honest — real load stacks on top.
- `projection-api` has no Stryker config (unchanged); the four mutated services carry the metrics wiring in `Program.cs`, same as the merged S-16b tracing code.
- Metric names verified against a live service: `http_server_request_duration_seconds{,_bucket,_count}`, label `http_response_status_code`, `dotnet_process_cpu_time_seconds_total`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #129
2026-07-24 08:31:41 +00:00

184 lines
7.7 KiB
YAML

name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
# Supersede stale runs: a new push to the same branch/PR cancels the previous run, so the runner's
# concurrency slots aren't spent on commits nobody is waiting for (refs #127).
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# Self-hosted runner — see docs/runbooks/ci.md for the runner setup.
# `uses:` are absolute, tag-pinned URLs (CLAUDE.md §8.7 / §15).
# Each job calls a `make` target — the same one developers run locally
# (`make ci`). The Makefile is the single source of truth; see docs/runbooks/ci.md.
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
# Cache the NuGet package store so each .NET job restores from disk, not the network. There are
# no lock files (so setup-dotnet's built-in cache doesn't apply); key on the project files. @v3
# avoids the GHES guard that breaks @v4 on Gitea (gitea-actions-gotchas.md); cache is best-effort
# — a miss just restores from the network. See issue #73.
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make lint
build:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make build
unit:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make unit
# Frontend (Nx/Angular) lane: install with pnpm, then Nx lint + test + build.
frontend:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/pnpm/action-setup@v4
with:
version: 11
- uses: https://github.com/actions/setup-node@v4
with:
node-version: '24'
cache: 'pnpm'
- run: make frontend
mutation:
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
- uses: https://github.com/actions/setup-dotnet@v4
with:
dotnet-version: '10.0.x'
- uses: https://github.com/actions/cache@v3
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('**/*.csproj') }}
restore-keys: |
nuget-${{ runner.os }}-
- run: make mutation
# Publish the Stryker HTML reports. `if: always()` uploads them even when the
# ratchet fails — that is exactly when you want to inspect the survivors.
# `continue-on-error` keeps the upload best-effort: the mutation *gate* is the
# ratchet (make mutation's exit code), not the report, so a Gitea artifact-backend
# 500 must not fail the job (gitea-actions-gotchas.md §4). Glob handles Stryker's
# non-deterministic StrykerOutput/<timestamp>/ dir. Pinned @v3: @v4's bundled
# @actions/artifact hard-aborts on non-github.com (GHES guard) — see the runbook.
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: acl-mutation-report
path: services/acl/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: event-subscriber-mutation-report
path: services/event-subscriber/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: domain-mutation-report
path: services/domain/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
- uses: https://github.com/actions/upload-artifact@v3
if: always()
continue-on-error: true
with:
name: bff-mutation-report
path: services/bff/StrykerOutput/**/reports/mutation-report.html
if-no-files-found: warn
# One stage for every check that needs the live stack. Booting OpenZaak once (instead
# of once per job) is the cheapest layout (issue #58). No setup-dotnet: the ACL test runs
# in a built image and everything reaches services by container IP. Needs Docker + egress
# (base images, nuget, selectielijst.openzaak.nl).
#
# `needs: [mutation]` is NOT a data dependency — it serialises the two memory-heavy jobs so
# they never co-schedule now the runner has capacity >1. A concurrent Stryker run + full-stack
# bring-up + Playwright browser on one host is what OOMs the e2e (commit d5e5fa2, #126). The
# light .NET/frontend jobs have no `needs`, so they still parallelise up to runner capacity.
# `if: !cancelled()` keeps verify-stack running even when the mutation ratchet fails (so we don't
# lose its signal) while still honouring run cancellation from the concurrency group above.
verify-stack:
needs: [mutation]
if: ${{ !cancelled() }}
runs-on: ubuntu-latest
steps:
- uses: https://github.com/actions/checkout@v4
# Bring the full stack up + wait for health — this also is the DoD "compose up
# reaches green health" smoke (it replaces the old compose-smoke job).
- name: Bring up the full stack & wait for health
run: make verify-up
- name: Observability backplane (Grafana + Tempo + Prometheus datasources)
run: OBS_TIMEOUT=180 make verify-observability
- name: ACL ↔ OpenZaak integration tests
run: make verify-acl
- name: OpenZaak → NRC notification delivery
run: make verify-nrc
- name: OpenZaak → NRC → Event Subscriber → projection-api
run: make verify-projection
- name: Domain → Flowable → ACL → OpenZaak
run: make verify-domain
- name: BFF → Keycloak + domain + projection
run: make verify-bff
- name: Distributed traces reach Tempo (one connected trace across services)
run: TRACING_TIMEOUT=120 make verify-tracing
- name: Golden-signal metrics scraped by Prometheus (/metrics on every service)
run: METRICS_TIMEOUT=120 make verify-metrics
- name: Self-service e2e (Playwright, login → submit → success)
run: make verify-e2e
# Log dump must precede teardown (which removes the containers).
- name: Dump container logs on failure
if: failure()
run: docker compose -f infra/docker-compose.yml logs --no-color --tail=100 oz-init openzaak nrc-init nrc-web nrc-celery nrc-beat flowable-db flowable-rest flowable-init keycloak acl bff domain projection-db event-subscriber projection-api self-service openbaar behandel tempo prometheus grafana 2>&1 || true
- name: Tear down
if: always()
run: make down