not
2125fb0cfd
## What & why S-19b-1. A write to the Objecten API now produces a **delivered** notification on the `objecten` kanaal in Open Notificaties. ADR-0028 switched Objecten's notifications off on purpose — there was no broker, worker, kanaal or abonnement, so wiring only the client side would have dropped every message on the floor. This slice builds the real path and turns it back on. - `objecten-celery` worker (mirrors `oz-celery`) + `CELERY_BROKER_URL`/`RESULT_BACKEND` on objecten-redis db 1 (db 0 is already the cache). `notifications_api_common` only *queues* the send; without a worker every register write is silently undelivered. - `nrc` service + `notifications_config` in Objecten's `setup_configuration`, reusing the `big-reference-seed` credential OpenZaak publishes with (NRC authorizes it via OpenZaak's AC, which grants it `heeft_alle_autorisaties` — no second credential needed). - The `objecten` kanaal in NRC's `setup_configuration`. The name is fixed by the Objects API (`NOTIFICATIONS_KANAAL`), not chosen here; publishing to an unregistered kanaal is exactly what the red check reported first. - `NOTIFICATIONS_DISABLED: "false"` in both compose files. - Writers address Objecten as `objecten.local` — see *Notes for reviewers*. - `make verify-objecten-notifications` — registers an abonnement on `objecten` pointing at a throwaway sink, writes a `RegisterRecord` exactly as the ACL does on approval, asserts the delivery. One assertion covering the whole chain: Objecten -> objecten-celery -> NRC -> nrc-beat -> callback. Wired into the CI `verify-stack` job and the summary table. **ADR-0029** records the decisions; ADR-0028's ceiling now points at it. Closes #152 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation (dc9ca2c, red at the first hop: `NRC POST /api/v1/abonnement -> 400 "Kanaal met deze naam bestaat niet."`). - [x] Implementation makes the test pass (4488962, + two fixes found by CI, below). - [x] Conventional Commits referencing the issue (`refs #152`). - [x] CI green — all six jobs ona5fd47e, including `verify-stack` end to end (e2e included). - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (`verify-stack`'s bring-up step). - [x] Docs updated — ADR-0029 added, ADR-0028's ceiling annotated, BACKLOG.md split. - [x] ADR added in `docs/architecture/`. - [x] Demo note in `docs/demo-script.md` if user-visible — n/a, infrastructure only; nothing consumes the kanaal until S-19b-2 (#153). ## Notes for reviewers **The one genuinely non-obvious bit: writers address Objecten as `objecten.local:8000`, not `objecten:8000`.** NRC types a notification's `hoofdObject`/`resourceUrl` as DRF `URLField`, so Django's `URLValidator` runs on them — and it rejects a **single-label** host. Objecten fills both from the object url DRF built with `request.build_absolute_uri`, i.e. *the Host the caller used*. Writing via the plain service name returns 201 and then fails every publish in the background, forever, with ``` 400 {"hoofdObject":["Voer een geldige URL in."],"resourceUrl":["Voer een geldige URL in."]} ``` So the `objecten` service carries an `objecten.local` network alias and every writer uses it — `Acl__Objecten__BaseUrl`, `ObjectenGatewayIntegrationTests`, this slice's verify driver. An alias rather than a bare dotted `SITE_DOMAIN` so the host still *resolves*: a subscriber following `resourceUrl` reaches the record, which S-19b-2 will do. Readers keep the plain name. Same class of constraint as ADR-0028's Objecttypen base-URL rule. **Ceiling, stated in the ADR:** nothing enforces the alias — a future writer using `objecten:8000` gets a 201 and silently no notification. If a second writer ever appears, rename the compose service rather than adding a lint. **Two CI-only failures on the way here**, both worth knowing: 1. `SITE_DOMAIN` was my first guess at the mechanism and is simply not what builds those URLs — dropped ind76abf2. 2. The check correlated the delivery on the `reference` inside the record it wrote. An NRC notification carries `kanaal`/`resource`/`kenmerken`/`hoofdObject`/`resourceUrl` and **never the record data**, so it correlates on the object URL now (a5fd47e). **Cost:** one more long-running container on the memory-tight runner. It inherits the capped `UWSGI_PROCESSES: "1"` env, which the celery command ignores; if `verify-stack` gets tight again, celery concurrency is the next knob. **Follow-up:** S-19b-2 (#153) sources the projection from these events. Nothing subscribes to the `objecten` kanaal in the product yet — only the verify check does.Reviewed-on: #154
332 lines
16 KiB
Makefile
332 lines
16 KiB
Makefile
# Developer + CI entrypoints.
|
|
#
|
|
# These targets are the single source of truth for the checks. The Gitea
|
|
# Actions workflow (.gitea/workflows/ci.yaml) invokes the SAME targets, so
|
|
# `make ci` locally runs exactly what the pipeline runs — no drift. Until a
|
|
# self-hosted runner is registered, `make ci` is the gate (see docs/runbooks/ci.md).
|
|
|
|
SLN := register-referentie.slnx
|
|
COMPOSE := infra/docker-compose.yml
|
|
# Long-running services with a healthcheck — the smoke polls these for readiness
|
|
# (infra/wait-healthy.sh). One-shot init jobs (oz-init, nrc-init, flowable-init)
|
|
# are not polled; they only need to have run. See docs/runbooks/gitea-actions-gotchas.md.
|
|
WAIT_SVCS := openzaak nrc-web acl bff domain event-subscriber projection-api self-service openbaar behandel beheer objecttypen objecten
|
|
# Config files (OpenZaak data.yaml, Keycloak realms, Flowable BPMN) are streamed
|
|
# into external named volumes via `docker cp` (infra/seed-config.sh) instead of
|
|
# bind-mounted, because bind mounts don't reach sibling containers on the
|
|
# containerized CI runner. SEED populates them; run it before every `up`. The
|
|
# volumes are `external`, so compose won't remove them — CFG_VOLS lists them for
|
|
# explicit teardown. See docs/runbooks/gitea-actions-gotchas.md.
|
|
SEED := bash infra/seed-config.sh
|
|
CFG_VOLS := rr-oz-config rr-nrc-config rr-kc-realms rr-fl-bpmn rr-objecttypen-config rr-objecten-config rr-registerrecord-config
|
|
# Local-only stack: same services but config is bind-mounted (no seed step), so a
|
|
# plain `docker compose -f infra/docker-compose.local.yml up` works on any local
|
|
# engine. This is the no-make / Windows-friendly path. See that file's header.
|
|
LOCAL_COMPOSE := infra/docker-compose.local.yml
|
|
OZ_COMPOSE := infra/openzaak/docker-compose.yml
|
|
OZ_BASE := http://localhost:8000
|
|
NRC_COMPOSE := infra/opennotificaties/docker-compose.yml
|
|
NRC_BASE := http://localhost:8001
|
|
KC_COMPOSE := infra/keycloak/docker-compose.yml
|
|
KC_BASE := http://localhost:8180
|
|
FL_COMPOSE := infra/flowable/docker-compose.yml
|
|
FL_BASE := http://localhost:8090/flowable-rest/service
|
|
STACK_FILES := -f $(OZ_COMPOSE) -f $(NRC_COMPOSE)
|
|
|
|
# On a rootless Podman dev box, point Docker CLI/Compose at the Podman socket —
|
|
# but only if that socket exists and DOCKER_HOST isn't already set, so real
|
|
# Docker hosts and CI runners are left untouched.
|
|
PODMAN_SOCK := /run/user/$(shell id -u)/podman/podman.sock
|
|
ifeq ($(wildcard $(PODMAN_SOCK)),$(PODMAN_SOCK))
|
|
ifeq ($(origin DOCKER_HOST),undefined)
|
|
export DOCKER_HOST := unix://$(PODMAN_SOCK)
|
|
endif
|
|
endif
|
|
|
|
.PHONY: ci lint build unit mutation frontend integration verify verify-up verify-acl verify-nrc verify-projection verify-bff verify-domain verify-observability verify-tracing verify-metrics verify-objecttypen verify-objecten verify-registerrecord verify-objecten-notifications verify-notifications smoke up down local verify-local local-down changelog openzaak-up openzaak-smoke openzaak-seed openzaak-down stack-up stack-smoke stack-down keycloak-up keycloak-smoke keycloak-down flowable-up flowable-smoke flowable-down help
|
|
|
|
## ci: run the full pipeline — lint, build, unit, mutation, frontend, verify (mirrors Gitea Actions)
|
|
## `verify` is the live-stack stage (full stack up once → ACL + notification checks).
|
|
ci: lint build unit mutation frontend verify
|
|
|
|
## frontend: install deps and run the Nx lint/test/build for the portals (pnpm + Node required)
|
|
# Tests run in their own phase, ahead of the build. The @angular/build:unit-test
|
|
# (Vitest) runner spawns a worker with a hard-coded 60s/90s startup timeout that is
|
|
# not configurable. When the ~5min production build shares the run-many pool, it
|
|
# starves that worker of CPU on constrained CI runners and Vitest fails with
|
|
# "Timeout waiting for worker to respond". Splitting the phases keeps tests off the
|
|
# heavy build's back so the worker starts well inside its window.
|
|
frontend:
|
|
pnpm install --frozen-lockfile
|
|
pnpm nx run-many -t lint test
|
|
pnpm nx run-many -t build
|
|
|
|
## lint: verify formatting (no changes)
|
|
lint:
|
|
dotnet format $(SLN) --verify-no-changes
|
|
|
|
## build: release build
|
|
build:
|
|
dotnet build $(SLN) -c Release
|
|
|
|
## unit: run unit tests (excludes the container-backed Integration lane)
|
|
# TRX per test project (→ TestResults/) feeds the CI per-service summary (#136); harmless locally.
|
|
unit:
|
|
dotnet test $(SLN) -c Release --filter "Category!=Integration" --logger trx --results-directory TestResults
|
|
|
|
## mutation: run the Stryker.NET ratchet on each service with branching logic (fails below baseline)
|
|
# Stryker is pinned as a local dotnet tool (.config/dotnet-tools.json); `tool restore`
|
|
# makes `make mutation` work from a fresh clone. Each service owns its config + break
|
|
# threshold (the ratchet, CLAUDE.md §5): each services/<svc>/stryker-config.json.
|
|
# Scores never regress below baseline.
|
|
mutation:
|
|
dotnet tool restore
|
|
cd services/acl && dotnet stryker
|
|
cd services/event-subscriber && dotnet stryker
|
|
cd services/domain && dotnet stryker
|
|
cd services/bff && dotnet stryker
|
|
|
|
## smoke: seed config, bring the whole stack up, wait for health-checked services, tear down
|
|
# SEED populates the external config volumes first (upstream images used verbatim;
|
|
# only our acl/bff are built). `up -d --build` starts EVERYTHING. Readiness is
|
|
# checked by infra/wait-healthy.sh polling the durable, health-checked services
|
|
# ($(WAIT_SVCS)) via `docker inspect` — portable across docker compose and
|
|
# podman-compose, and needing no `--wait` flag or host port access. The one-shots
|
|
# (oz-init, flowable-init) aren't polled; they just need to have run.
|
|
smoke:
|
|
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
|
|
docker compose -f $(COMPOSE) up -d --build
|
|
bash -c 'WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS); rc=$$?; docker compose -f $(COMPOSE) down --volumes; docker volume rm -f $(CFG_VOLS) >/dev/null 2>&1; exit $$rc'
|
|
|
|
## up: seed config volumes and start the full stack (use instead of bare
|
|
## `docker compose up`, which can't self-seed the external config volumes)
|
|
up:
|
|
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
|
|
docker compose -f $(COMPOSE) up -d --build
|
|
|
|
## down: stop and remove the local stack (incl. the external config volumes)
|
|
down:
|
|
docker compose -f $(COMPOSE) down --volumes
|
|
-docker volume rm -f $(CFG_VOLS)
|
|
|
|
## local: bring up the bind-mount stack (no seed step) and wait for health
|
|
## (Windows / no-make users: run `docker compose -f infra/docker-compose.local.yml up -d --build` directly)
|
|
local:
|
|
docker compose -f $(LOCAL_COMPOSE) up -d --build
|
|
WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS)
|
|
|
|
## verify-local: acceptance check for the local stack (S-B04) — a fresh `make local` completes the
|
|
## whole flow (zaaktype seeded + DMN deployed + NRC abonnement) with NO manual seeding.
|
|
verify-local:
|
|
bash infra/run-local-flow-check.sh
|
|
|
|
## local-down: stop and remove the bind-mount stack
|
|
local-down:
|
|
docker compose -f $(LOCAL_COMPOSE) down --volumes
|
|
|
|
## changelog: regenerate CHANGELOG.md from Conventional Commits (git-cliff)
|
|
changelog:
|
|
git-cliff --output CHANGELOG.md
|
|
|
|
# ── ZGW verification ───────────────────────────────────────────────────────
|
|
# On the single runner CI jobs run sequentially, so the OpenZaak-dependent checks
|
|
# share ONE full-stack bring-up: the `verify-stack` CI job runs `verify-up` then
|
|
# `verify-acl` + `verify-nrc` as steps against the same stack (issue #58). The
|
|
# check logic lives in stack-agnostic runners that reach services by container IP
|
|
# (gitea-actions-gotchas.md §5/§6); `integration` / `verify-notifications` are local
|
|
# convenience wrappers that bring up a lighter stack and call the same runners.
|
|
|
|
## verify-up: bring the FULL stack up and wait for health (CI verify-stack step 1;
|
|
## subsumes the old compose-smoke health gate — the DoD "up reaches green" check).
|
|
verify-up:
|
|
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
|
|
docker compose -f $(COMPOSE) up -d --build
|
|
WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS)
|
|
|
|
## verify-acl: ACL ↔ OpenZaak integration tests against the already-running stack.
|
|
verify-acl:
|
|
bash infra/run-acl-integration.sh
|
|
|
|
## verify-nrc: OpenZaak → NRC notification delivery against the already-running stack.
|
|
verify-nrc:
|
|
bash infra/run-notification-check.sh
|
|
|
|
## verify-projection: OpenZaak → NRC → Event Subscriber → projection-api end-to-end (S-06),
|
|
## against the already-running stack.
|
|
verify-projection:
|
|
bash infra/run-projection-check.sh
|
|
|
|
## verify-domain: domain → Flowable → ACL → OpenZaak end-to-end (S-05), against the
|
|
## already-running stack. Recreates the acl service to inject the seeded zaaktype URL.
|
|
verify-domain:
|
|
bash infra/run-domain-check.sh
|
|
|
|
## verify-bff: BFF end-to-end (S-07) against the up stack — token validation on self-service
|
|
## + anonymous public-safe openbaar register (ADR-0010).
|
|
verify-bff:
|
|
bash infra/run-bff-check.sh
|
|
|
|
## verify-e2e: walking-skeleton Playwright e2e (S-08d) against the up stack — DigiD login →
|
|
## submit → confirmation, driven inside the compose network.
|
|
verify-e2e:
|
|
bash infra/run-e2e-check.sh
|
|
|
|
## verify-observability: assert the observability backplane (Grafana + provisioned Tempo &
|
|
## Prometheus datasources) is live, against the already-running stack (S-16a).
|
|
verify-observability:
|
|
bash infra/run-observability-check.sh
|
|
|
|
## verify-tracing: assert one connected distributed trace spans the .NET services in Tempo
|
|
## (S-16b), against the already-running stack.
|
|
verify-tracing:
|
|
bash infra/run-tracing-check.sh
|
|
|
|
## verify-metrics: assert the services expose /metrics and Prometheus scrapes the golden
|
|
## signals (S-16c), against the already-running stack.
|
|
verify-metrics:
|
|
bash infra/run-metrics-check.sh
|
|
|
|
## verify-objecttypen: assert the Objecttypen API is up + its static token authenticates
|
|
## (S-18a), against the already-running stack.
|
|
verify-objecttypen:
|
|
bash infra/run-objecttypen-check.sh
|
|
|
|
## verify-objecten: assert the Objecten API is up + its static token authenticates and it
|
|
## trusts the Objecttypen API (S-18b), against the already-running stack.
|
|
verify-objecten:
|
|
bash infra/run-objecten-check.sh
|
|
|
|
## verify-registerrecord: assert the RegisterRecord objecttype is registered + published in the
|
|
## Objecttypen API (S-18c), against the already-running stack.
|
|
verify-registerrecord:
|
|
bash infra/run-registerrecord-check.sh
|
|
|
|
## verify-objecten-notifications: assert a RegisterRecord write in Objecten is DELIVERED as an
|
|
## `objecten` notification via NRC (S-19b-1), against the already-running stack.
|
|
verify-objecten-notifications:
|
|
bash infra/run-objecten-notifications-check.sh
|
|
|
|
## verify: local mirror of the CI verify-stack job — full stack up once, all checks,
|
|
## tear down (always). For fast single-concern local iteration use `integration`
|
|
## (oz-only) or `verify-notifications` (oz+nrc) instead.
|
|
verify:
|
|
$(SEED) oz nrc kc fl objecttypen objecten registerrecord
|
|
docker compose -f $(COMPOSE) up -d --build
|
|
@bash -c 'set -e; rc=0; \
|
|
WAIT_TIMEOUT=420 bash infra/wait-healthy.sh $(WAIT_SVCS) \
|
|
&& bash infra/run-acl-integration.sh \
|
|
&& bash infra/run-notification-check.sh \
|
|
&& bash infra/run-projection-check.sh \
|
|
&& bash infra/run-objecten-notifications-check.sh \
|
|
&& bash infra/run-domain-check.sh \
|
|
&& bash infra/run-bff-check.sh \
|
|
&& bash infra/run-e2e-check.sh || rc=$$?; \
|
|
docker compose -f $(COMPOSE) down --volumes >/dev/null 2>&1; \
|
|
docker volume rm -f $(CFG_VOLS) >/dev/null 2>&1; \
|
|
exit $$rc'
|
|
|
|
## integration: local convenience — ACL integration test against a throwaway
|
|
## OpenZaak-only stack (fast iteration). CI uses verify-acl on the shared stack.
|
|
integration:
|
|
bash infra/run-integration.sh
|
|
|
|
## openzaak-up: start the OpenZaak stack (migrations run on first start)
|
|
openzaak-up:
|
|
$(SEED) oz
|
|
docker compose -f $(OZ_COMPOSE) up -d
|
|
|
|
## openzaak-smoke: start OpenZaak, then assert it is up with auth enforced
|
|
openzaak-smoke: openzaak-up
|
|
@bash -c 'set -e; \
|
|
echo "waiting for OpenZaak to respond..."; \
|
|
for i in $$(seq 1 60); do \
|
|
code=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/zaken/api/v1/zaken || true); \
|
|
[ -n "$$code" ] && [ "$$code" != "000" ] && break; sleep 3; \
|
|
done; \
|
|
echo "GET /zaken/api/v1/zaken (unauth) -> $$code (expect 403, auth enforced)"; test "$$code" = "403"; \
|
|
admin=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/admin/); \
|
|
echo "GET /admin/ -> $$admin (expect 302)"; test "$$admin" = "302"; \
|
|
root=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/zaken/api/v1/); \
|
|
echo "GET /zaken/api/v1/ -> $$root (expect 200)"; test "$$root" = "200"; \
|
|
echo "OpenZaak smoke OK"'
|
|
|
|
## openzaak-seed: bring OpenZaak up and seed the BIG catalogus (idempotent)
|
|
openzaak-seed: openzaak-up
|
|
@bash -c 'for i in $$(seq 1 50); do \
|
|
c=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/catalogi/api/v1/ || true); \
|
|
[ "$$c" = "200" ] && break; sleep 3; done; echo "OpenZaak ready ($$c)"'
|
|
python3 infra/openzaak/seed_catalogus.py
|
|
|
|
## openzaak-down: stop and remove the OpenZaak stack (wipes data)
|
|
openzaak-down:
|
|
docker compose -f $(OZ_COMPOSE) down --volumes
|
|
-docker volume rm -f rr-oz-config
|
|
|
|
## verify-notifications: local convenience — OpenZaak → NRC notification delivery
|
|
## against a throwaway oz+nrc stack (S-01-c). CI uses verify-nrc on the shared stack.
|
|
verify-notifications:
|
|
bash infra/verify-notifications.sh
|
|
|
|
## stack-up: start OpenZaak + Open Notificaties together (shared network), with
|
|
## OpenZaak publishing notifications to NRC (S-01-c).
|
|
stack-up:
|
|
$(SEED) oz nrc
|
|
OZ_NOTIFICATIONS_DISABLED=false docker compose $(STACK_FILES) up -d
|
|
|
|
## stack-smoke: start both, assert OpenZaak (403/302/200) and NRC (302) are reachable
|
|
stack-smoke: stack-up
|
|
@bash -c 'set -e; \
|
|
echo "waiting for OpenZaak + Open Notificaties..."; \
|
|
for i in $$(seq 1 60); do \
|
|
oz=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/admin/ || true); \
|
|
nrc=$$(curl -s -o /dev/null -w "%{http_code}" $(NRC_BASE)/admin/ || true); \
|
|
[ "$$oz" = "302" ] && [ "$$nrc" = "302" ] && break; sleep 3; done; \
|
|
z=$$(curl -s -o /dev/null -w "%{http_code}" $(OZ_BASE)/zaken/api/v1/zaken); \
|
|
echo "OpenZaak /zaken (unauth) -> $$z (expect 403)"; test "$$z" = "403"; \
|
|
echo "OpenZaak /admin/ -> $$oz (expect 302)"; test "$$oz" = "302"; \
|
|
echo "Open Notificaties /admin/-> $$nrc (expect 302)"; test "$$nrc" = "302"; \
|
|
echo "stack smoke OK"'
|
|
|
|
## stack-down: stop and remove both stacks (wipes data)
|
|
stack-down:
|
|
docker compose $(STACK_FILES) down --volumes
|
|
-docker volume rm -f rr-oz-config rr-nrc-config
|
|
|
|
## keycloak-up: start Keycloak with the four imported realms
|
|
keycloak-up:
|
|
$(SEED) kc
|
|
docker compose -f $(KC_COMPOSE) up -d
|
|
|
|
## keycloak-smoke: start Keycloak, then verify each realm logs in + returns its claim
|
|
keycloak-smoke: keycloak-up
|
|
@bash -c 'for i in $$(seq 1 60); do \
|
|
c=$$(curl -s -o /dev/null -w "%{http_code}" $(KC_BASE)/realms/digid/.well-known/openid-configuration || true); \
|
|
[ "$$c" = "200" ] && break; sleep 3; done; echo "Keycloak ready ($$c)"'
|
|
python3 infra/keycloak/check_realms.py
|
|
|
|
## keycloak-down: stop and remove Keycloak
|
|
keycloak-down:
|
|
docker compose -f $(KC_COMPOSE) down --volumes
|
|
-docker volume rm -f rr-kc-realms
|
|
|
|
## flowable-up: start Flowable (deploys registratie.bpmn on boot)
|
|
flowable-up:
|
|
$(SEED) fl
|
|
docker compose -f $(FL_COMPOSE) up -d
|
|
|
|
## flowable-smoke: start Flowable, then verify a started instance waits on the external task
|
|
flowable-smoke: flowable-up
|
|
@bash -c 'for i in $$(seq 1 80); do \
|
|
c=$$(curl -s -o /dev/null -w "%{http_code}" -u rest-admin:test $(FL_BASE)/repository/process-definitions?key=registratie || true); \
|
|
[ "$$c" = "200" ] && break; sleep 3; done; echo "Flowable ready ($$c)"'
|
|
python3 infra/flowable/verify.py
|
|
|
|
## flowable-down: stop and remove Flowable
|
|
flowable-down:
|
|
docker compose -f $(FL_COMPOSE) down --volumes
|
|
-docker volume rm -f rr-fl-bpmn
|
|
|
|
## help: list available targets
|
|
help:
|
|
@grep -E '^## ' $(MAKEFILE_LIST) | sed 's/^## //'
|