feat(infra): Objecten publishes register events to NRC (closes #152) (#154)
CI / build (push) Successful in 1m14s
CI / lint (push) Successful in 1m28s
CI / unit (push) Successful in 1m28s
CI / frontend (push) Successful in 3m8s
CI / mutation (push) Successful in 6m35s
CI / verify-stack (push) Successful in 9m27s

## What & why

S-19b-1. A write to the Objecten API now produces a **delivered** notification on the
`objecten` kanaal in Open Notificaties. ADR-0028 switched Objecten's notifications off on
purpose — there was no broker, worker, kanaal or abonnement, so wiring only the client side
would have dropped every message on the floor. This slice builds the real path and turns it
back on.

- `objecten-celery` worker (mirrors `oz-celery`) + `CELERY_BROKER_URL`/`RESULT_BACKEND` on
  objecten-redis db 1 (db 0 is already the cache). `notifications_api_common` only *queues*
  the send; without a worker every register write is silently undelivered.
- `nrc` service + `notifications_config` in Objecten's `setup_configuration`, reusing the
  `big-reference-seed` credential OpenZaak publishes with (NRC authorizes it via OpenZaak's
  AC, which grants it `heeft_alle_autorisaties` — no second credential needed).
- The `objecten` kanaal in NRC's `setup_configuration`. The name is fixed by the Objects API
  (`NOTIFICATIONS_KANAAL`), not chosen here; publishing to an unregistered kanaal is exactly
  what the red check reported first.
- `NOTIFICATIONS_DISABLED: "false"` in both compose files.
- Writers address Objecten as `objecten.local` — see *Notes for reviewers*.
- `make verify-objecten-notifications` — registers an abonnement on `objecten` pointing at a
  throwaway sink, writes a `RegisterRecord` exactly as the ACL does on approval, asserts the
  delivery. One assertion covering the whole chain: Objecten -> objecten-celery -> NRC ->
  nrc-beat -> callback. Wired into the CI `verify-stack` job and the summary table.

**ADR-0029** records the decisions; ADR-0028's ceiling now points at it.

Closes #152

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing test committed before the implementation (dc9ca2c, red at the first hop:
      `NRC POST /api/v1/abonnement -> 400 "Kanaal met deze naam bestaat niet."`).
- [x] Implementation makes the test pass (4488962, + two fixes found by CI, below).
- [x] Conventional Commits referencing the issue (`refs #152`).
- [x] CI green — all six jobs on a5fd47e, including `verify-stack` end to end (e2e included).
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step).
- [x] Docs updated — ADR-0029 added, ADR-0028's ceiling annotated, BACKLOG.md split.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` if user-visible — n/a, infrastructure only; nothing
      consumes the kanaal until S-19b-2 (#153).

## Notes for reviewers

**The one genuinely non-obvious bit: writers address Objecten as `objecten.local:8000`, not
`objecten:8000`.** NRC types a notification's `hoofdObject`/`resourceUrl` as DRF `URLField`,
so Django's `URLValidator` runs on them — and it rejects a **single-label** host. Objecten
fills both from the object url DRF built with `request.build_absolute_uri`, i.e. *the Host
the caller used*. Writing via the plain service name returns 201 and then fails every
publish in the background, forever, with

```
400 {"hoofdObject":["Voer een geldige URL in."],"resourceUrl":["Voer een geldige URL in."]}
```

So the `objecten` service carries an `objecten.local` network alias and every writer uses it
— `Acl__Objecten__BaseUrl`, `ObjectenGatewayIntegrationTests`, this slice's verify driver.
An alias rather than a bare dotted `SITE_DOMAIN` so the host still *resolves*: a subscriber
following `resourceUrl` reaches the record, which S-19b-2 will do. Readers keep the plain
name. Same class of constraint as ADR-0028's Objecttypen base-URL rule.

**Ceiling, stated in the ADR:** nothing enforces the alias — a future writer using
`objecten:8000` gets a 201 and silently no notification. If a second writer ever appears,
rename the compose service rather than adding a lint.

**Two CI-only failures on the way here**, both worth knowing:
1. `SITE_DOMAIN` was my first guess at the mechanism and is simply not what builds those
   URLs — dropped in d76abf2.
2. The check correlated the delivery on the `reference` inside the record it wrote. An NRC
   notification carries `kanaal`/`resource`/`kenmerken`/`hoofdObject`/`resourceUrl` and
   **never the record data**, so it correlates on the object URL now (a5fd47e).

**Cost:** one more long-running container on the memory-tight runner. It inherits the capped
`UWSGI_PROCESSES: "1"` env, which the celery command ignores; if `verify-stack` gets tight
again, celery concurrency is the next knob.

**Follow-up:** S-19b-2 (#153) sources the projection from these events. Nothing subscribes to
the `objecten` kanaal in the product yet — only the verify check does.Reviewed-on: #154
This commit was merged in pull request #154.
This commit is contained in:
not
2026-08-28 10:11:54 +00:00
parent 0cd70ae8c3
commit 2125fb0cfd
12 changed files with 438 additions and 22 deletions
+32 -7
View File
@@ -341,7 +341,8 @@ services:
# Objecten holds the register, OpenZaak holds the process (S-19a, ADR-0028). Both APIs take a
# static token, not a ZGW JWT. The objecttype URL is assigned at seed time, so the ACL resolves
# it by name — lazily, on the first approval, so no depends_on is needed here.
Acl__Objecten__BaseUrl: http://objecten:8000/
# Dotted host on purpose — see the `objecten.local` alias below (ADR-0029).
Acl__Objecten__BaseUrl: http://objecten.local:8000/
Acl__Objecten__Token: ${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}
Acl__Objecten__ObjecttypenBaseUrl: http://objecttypen:8000/
Acl__Objecten__ObjecttypenToken: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
@@ -691,12 +692,14 @@ services:
CACHE_AXES: objecten-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
# S-19a: Objecten refuses every write while its Notificaties config is absent
# (notifications_api_common raises rather than skipping, so POST /objects 500s). Objecten →
# NRC is not wired yet — there is no broker, worker, kanaal or abonnement for it — so turn
# notifications off rather than fake a delivery path that silently drops every message.
# S-19b (#150) sources the projection from Objecten and turns this back on for real.
NOTIFICATIONS_DISABLED: "true"
CELERY_BROKER_URL: redis://objecten-redis:6379/1
CELERY_RESULT_BACKEND: redis://objecten-redis:6379/1
# Publish register-record events to NRC on the `objecten` kanaal (S-19b-1, ADR-0029). The NRC
# service + notifications_config are provisioned by setup_configuration
# (infra/objecten/setup_configuration/data.yaml), and objecten-celery below actually sends
# them — notifications_api_common only queues the task. See ADR-0028 for why S-19a left this
# off until all four pieces existed.
NOTIFICATIONS_DISABLED: "false"
RUN_SETUP_CONFIG: "true"
command: /setup_configuration.sh
volumes:
@@ -721,6 +724,28 @@ services:
start_period: 30s
ports:
- "8021:8000"
depends_on:
objecten-init:
condition: service_completed_successfully
networks:
cg:
# Objecten reflects the *request* Host into the `url` it returns, and
# notifications_api_common publishes that url as the notification's hoofdObject /
# resourceUrl — which NRC types as a URLField, and Django's URLValidator rejects a
# single-label host ("Voer een geldige URL in."). So every caller whose writes must be
# notified addresses Objecten by this dotted alias instead of `objecten` (ADR-0029).
# Reads are unaffected and still use the plain service name.
aliases:
- objecten.local
# The celery worker that actually delivers Objecten's notifications to NRC (S-19b-1, ADR-0029).
# notifications_api_common only schedules the send on transaction commit; without a worker the
# task sits in redis forever and every register write is silently undelivered. Mirrors oz-celery.
# No beat: Objecten is a publisher, not a subscriber — nrc-beat drains the delivery queue.
objecten-celery:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env-local
command: /celery_worker.sh
depends_on:
objecten-init:
condition: service_completed_successfully
+32 -7
View File
@@ -326,7 +326,8 @@ services:
# Objecten holds the register, OpenZaak holds the process (S-19a, ADR-0028). Both APIs take a
# static token, not a ZGW JWT. The objecttype URL is assigned at seed time, so the ACL resolves
# it by name — lazily, on the first approval, so no depends_on is needed here.
Acl__Objecten__BaseUrl: http://objecten:8000/
# Dotted host on purpose — see the `objecten.local` alias below (ADR-0029).
Acl__Objecten__BaseUrl: http://objecten.local:8000/
Acl__Objecten__Token: ${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}
Acl__Objecten__ObjecttypenBaseUrl: http://objecttypen:8000/
Acl__Objecten__ObjecttypenToken: ${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}
@@ -717,12 +718,14 @@ services:
CACHE_AXES: objecten-redis:6379/0
DISABLE_2FA: "true"
OTEL_SDK_DISABLED: "true"
# S-19a: Objecten refuses every write while its Notificaties config is absent
# (notifications_api_common raises rather than skipping, so POST /objects 500s). Objecten →
# NRC is not wired yet — there is no broker, worker, kanaal or abonnement for it — so turn
# notifications off rather than fake a delivery path that silently drops every message.
# S-19b (#150) sources the projection from Objecten and turns this back on for real.
NOTIFICATIONS_DISABLED: "true"
CELERY_BROKER_URL: redis://objecten-redis:6379/1
CELERY_RESULT_BACKEND: redis://objecten-redis:6379/1
# Publish register-record events to NRC on the `objecten` kanaal (S-19b-1, ADR-0029). The NRC
# service + notifications_config are provisioned by setup_configuration
# (infra/objecten/setup_configuration/data.yaml), and objecten-celery below actually sends
# them — notifications_api_common only queues the task. See ADR-0028 for why S-19a left this
# off until all four pieces existed.
NOTIFICATIONS_DISABLED: "false"
RUN_SETUP_CONFIG: "true"
command: /setup_configuration.sh
# data.yaml is streamed into this external volume by infra/seed-config.sh before start.
@@ -750,6 +753,28 @@ services:
start_period: 30s
ports:
- "8021:8000"
depends_on:
objecten-init:
condition: service_completed_successfully
networks:
cg:
# Objecten reflects the *request* Host into the `url` it returns, and
# notifications_api_common publishes that url as the notification's hoofdObject /
# resourceUrl — which NRC types as a URLField, and Django's URLValidator rejects a
# single-label host ("Voer een geldige URL in."). So every caller whose writes must be
# notified addresses Objecten by this dotted alias instead of `objecten` (ADR-0029).
# Reads are unaffected and still use the plain service name.
aliases:
- objecten.local
# The celery worker that actually delivers Objecten's notifications to NRC (S-19b-1, ADR-0029).
# notifications_api_common only schedules the send on transaction commit; without a worker the
# task sits in redis forever and every register write is silently undelivered. Mirrors oz-celery.
# No beat: Objecten is a publisher, not a subscriber — nrc-beat drains the delivery queue.
objecten-celery:
image: docker.io/maykinmedia/objects-api:${OBJECTS_TAG:-3.4.0}
environment: *objecten-env
command: /celery_worker.sh
depends_on:
objecten-init:
condition: service_completed_successfully
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env python3
"""S-19b-1 (#152): driver for the Objecten → NRC notification check.
Registers an abonnement on the `objecten` kanaal pointing at the webhook sink, then writes a
RegisterRecord object exactly as the ACL's ObjectenGateway does (S-19a). The caller
(run-objecten-notifications-check.sh) watches the sink for the delivery — this only sets it up,
and prints `OBJECT_URL <url>` for the caller to grep on.
Delivery exercises the whole chain: Objecten → its celery worker → NRC → nrc-beat → the callback.
Anything missing (broker, worker, kanaal, notifications config) shows up as a non-delivery.
Stdlib only so it runs in a bare python:3-slim container on the compose network.
"""
import base64
import hashlib
import hmac
import json
import os
import sys
import time
import urllib.error
import urllib.request
OBJECTEN = os.environ["OBJECTEN"] # http://objecten:8000
OBJECTEN_TOKEN = os.environ["OBJECTEN_TOKEN"]
OBJECTTYPEN = os.environ["OBJECTTYPEN"] # http://objecttypen:8000
OBJECTTYPEN_TOKEN = os.environ["OBJECTTYPEN_TOKEN"]
NRC_BASE = os.environ["NRC_BASE"] # http://<nrc-ip>:8000
SINK_CALLBACK = os.environ["SINK_CALLBACK"] # http://<sink-ip>:9000/
SINK_AUTH = os.environ["SINK_AUTH"]
CLIENT_ID = os.environ.get("NRC_CLIENT_ID", "big-reference-seed")
SECRET = os.environ.get("NRC_SECRET", "insecure-dev-secret-change-me")
KANAAL = "objecten"
def mint():
"""The HS256 JWT NRC expects (same shape as infra/local/register-abonnement.py)."""
def seg(d):
return base64.urlsafe_b64encode(json.dumps(d).encode()).rstrip(b"=")
payload = seg({
"iss": CLIENT_ID, "iat": int(time.time()), "client_id": CLIENT_ID,
"user_id": CLIENT_ID, "user_representation": CLIENT_ID,
})
signing_input = seg({"typ": "JWT", "alg": "HS256"}) + b"." + payload
signature = base64.urlsafe_b64encode(
hmac.new(SECRET.encode(), signing_input, hashlib.sha256).digest()).rstrip(b"=")
return (signing_input + b"." + signature).decode()
def nrc(method, url, body=None):
"""Call NRC. `url` may be a path or an absolute URL (the list returns absolute ones)."""
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(
url if url.startswith("http") else f"{NRC_BASE}{url}", data=data, method=method,
headers={"Authorization": f"Bearer {mint()}", "Content-Type": "application/json"})
try:
with urllib.request.urlopen(req, timeout=15) as r:
return json.load(r) if r.length != 0 else {}
except urllib.error.HTTPError as e:
# The body carries the reason (e.g. an unregistered kanaal); the status alone does not.
raise SystemExit(f"FAIL — NRC {method} {url}{e.code}: {e.read().decode(errors='replace')[:400]}")
def token_api(base, token, method, path, body=None, crs=False):
data = json.dumps(body).encode() if body is not None else None
headers = {"Authorization": f"Token {token}"}
if body is not None:
headers["Content-Type"] = "application/json"
if crs:
headers["Accept-Crs"] = "EPSG:4326"
if body is not None:
headers["Content-Crs"] = "EPSG:4326"
req = urllib.request.Request(f"{base}{path}", data=data, method=method, headers=headers)
with urllib.request.urlopen(req, timeout=15) as r:
return json.load(r) if r.length != 0 else {}
def subscribe():
"""Register an abonnement on the objecten kanaal, replacing a stale one for the same callback."""
# NRC returns a bare list here, not a paginated envelope.
for existing in nrc("GET", "/api/v1/abonnement") or []:
if existing.get("callbackUrl") == SINK_CALLBACK:
nrc("DELETE", existing["url"])
nrc("POST", "/api/v1/abonnement", {
"callbackUrl": SINK_CALLBACK,
"auth": SINK_AUTH,
"kanalen": [{"naam": KANAAL, "filters": {}}],
})
print(f">> abonnement on '{KANAAL}' -> {SINK_CALLBACK}")
def objecttype_url():
results = token_api(OBJECTTYPEN, OBJECTTYPEN_TOKEN, "GET", "/api/v2/objecttypes").get("results", [])
match = next((o for o in results if o.get("name") == "RegisterRecord"), None)
if not match:
print("FAIL — no RegisterRecord objecttype in Objecttypen", file=sys.stderr)
raise SystemExit(1)
return match["url"]
def main():
subscribe()
reference = f"NOTIF-{int(time.time())}"
created = token_api(OBJECTEN, OBJECTEN_TOKEN, "POST", "/api/v2/objects", {
"type": objecttype_url(),
"record": {
"typeVersion": 1,
"data": {"id": f"zaak-{reference}", "status": "INGESCHREVEN", "reference": reference},
"startAt": time.strftime("%Y-%m-%d"),
},
}, crs=True)
print(f">> wrote RegisterRecord {created['url']}")
# An NRC notification carries no record data — only hoofdObject/resourceUrl — so the object
# URL, not the reference in its data, is what the caller can correlate the delivery on.
print(f"OBJECT_URL {created['url']}")
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -18,6 +18,16 @@ zgw_consumers:
auth_type: api_key
header_key: Authorization
header_value: Token 0123456789abcdef0123456789abcdef01234567
# (1b) The NRC Objecten publishes register-record events to (S-19b-1, ADR-0029). Same shape and
# same big-reference-seed credential OpenZaak publishes with — NRC verifies the JWT and
# authorizes it via OpenZaak's AC, which grants that client heeft_alle_autorisaties.
- identifier: nrc
label: Open Notificaties
api_type: nrc
api_root: http://nrc-web:8000/api/v1/
auth_type: zgw
client_id: big-reference-seed
secret: insecure-dev-secret-change-me
# (2) Permit the RegisterRecord objecttype (S-19a). Objecten refuses to store an object whose
# objecttype it has not been configured with ("ObjectType with url=… is not configured"), and it
@@ -40,3 +50,10 @@ tokenauth:
email: admin@localhost
organization: Respellion
is_superuser: true
# (4) Point Objecten's notifications at that NRC service (S-19b-1, ADR-0029). Requires
# NOTIFICATIONS_DISABLED=false plus a celery broker + worker — without the worker the message is
# queued and never sent, which is exactly the half-wired state S-19a refused to ship (ADR-0028).
notifications_config_enable: true
notifications_config:
notifications_api_service_identifier: nrc
@@ -29,7 +29,9 @@ autorisaties_api_config_enable: true
autorisaties_api:
authorizations_api_service_identifier: openzaak-ac
# 4. The kanaal OpenZaak publishes zaak events on.
# 4. The kanalen publishers announce on: `zaken` (OpenZaak) and `objecten` (Objecten, S-19b-1).
# Both authenticate with the big-reference-seed credential above, which OpenZaak's AC grants
# heeft_alle_autorisaties — so no separate publisher authorization is needed for Objecten.
notifications_kanalen_config_enable: true
notifications_kanalen_config:
items:
@@ -39,3 +41,11 @@ notifications_kanalen_config:
- bronorganisatie
- zaaktype
- vertrouwelijkheidaanduiding
# 5. The kanaal Objecten publishes register-record events on (S-19b-1, ADR-0029). Its name is
# fixed by the Objects API itself (NOTIFICATIONS_KANAAL = "objecten"), not chosen here. The
# filter set matches what the Objects API sends as kenmerken, so an abonnement can narrow by
# objecttype rather than receiving every object write in the register.
- naam: objecten
documentatie_link: https://objects-and-objecttypes-api.readthedocs.io/
filters:
- object_type
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
#
# S-19b-1 (#152): verify the Objecten → NRC notification path against an ALREADY-RUNNING full
# stack. Registers an abonnement on the `objecten` kanaal pointing at a throwaway webhook sink,
# writes a RegisterRecord object (exactly as the ACL does on approval, S-19a), and asserts the sink
# receives the notification.
#
# This is the whole publish chain in one assertion: Objecten → its celery worker → NRC → nrc-beat →
# the subscriber callback. S-19a deliberately left it disconnected (ADR-0028); this proves it is
# connected for real, rather than merely configured.
#
# All in-network, reaching services by container IP (a single-label host isn't URL-valid for NRC's
# callbackUrl validator; the runner can't reach published ports — gitea-actions-gotchas.md §5/§6).
# EXCEPT Objecttypen, which must be reached by SERVICE NAME: it echoes the request Host into the
# objecttype `url` and Objecten only accepts the one matching its configured api_root (ADR-0028);
# and Objecten, reached by its `objecten.local` alias because it reflects the request Host into the
# notification's hoofdObject/resourceUrl, which NRC validates as a URL (ADR-0029).
#
# Does NOT manage the stack lifecycle, but cleans up the sink/driver it creates.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SINK_AUTH="Bearer objecten-notification-sink-token"
cleanup() { docker rm -f rr-osink rr-overify >/dev/null 2>&1 || true; }
trap cleanup EXIT
ip() { docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$1"; }
# Anchored on the compose replica suffix so they don't also match objecten-db / objecten-redis.
obj="$(docker ps -q --filter 'name=objecten[-_][0-9]+$' | head -1)"
nrc="$(docker ps -q --filter 'name=nrc-web' | head -1)"
[ -n "$obj" ] || { echo "ERROR: no running objecten container — bring the stack up first" >&2; exit 1; }
[ -n "$nrc" ] || { echo "ERROR: no running nrc-web container — bring the stack up first" >&2; exit 1; }
net="$(docker inspect -f '{{range $k,$_ := .NetworkSettings.Networks}}{{$k}}{{"\n"}}{{end}}' "$obj" | head -1)"
nrc_ip="$(ip "$nrc")"
echo ">> network=$net nrc=$nrc_ip"
echo ">> starting the webhook sink"
docker rm -f rr-osink >/dev/null 2>&1 || true
sink="$(docker create --network "$net" --name rr-osink -e "EXPECTED_AUTH=$SINK_AUTH" \
python:3-slim python /sink.py)"
docker cp "$here/notification-sink.py" "$sink:/sink.py" >/dev/null
docker start "$sink" >/dev/null
sleep 1
sink_ip="$(ip rr-osink)"
echo ">> sink at $sink_ip:9000"
echo ">> registering the abonnement + writing a RegisterRecord"
docker rm -f rr-overify >/dev/null 2>&1 || true
drv="$(docker create --network "$net" --name rr-overify \
-e "OBJECTEN=http://objecten.local:8000" \
-e "OBJECTEN_TOKEN=${OBJECTEN_TOKEN:-1234567890abcdef1234567890abcdef12345678}" \
-e "OBJECTTYPEN=http://objecttypen:8000" \
-e "OBJECTTYPEN_TOKEN=${OBJECTTYPEN_TOKEN:-0123456789abcdef0123456789abcdef01234567}" \
-e "NRC_BASE=http://$nrc_ip:8000" \
-e "SINK_CALLBACK=http://$sink_ip:9000/" -e "SINK_AUTH=$SINK_AUTH" \
python:3-slim python /driver.py)"
docker cp "$here/objecten-notifications-check.py" "$drv:/driver.py" >/dev/null
docker start -a "$drv"
object_url="$(docker logs rr-overify 2>/dev/null | sed -n 's/^OBJECT_URL //p' | head -1)"
docker rm -f rr-overify >/dev/null
[ -n "$object_url" ] || { echo "FAIL — the driver did not write a RegisterRecord" >&2; exit 1; }
echo ">> wrote $object_url"
# Correlate on the object URL: a notification carries hoofdObject/resourceUrl, never the record
# data, so the reference inside the record is not in the delivered message.
echo ">> waiting for the notification to reach the sink"
for _ in $(seq 1 "${NOTIFICATION_TRIES:-40}"); do
if docker logs rr-osink 2>&1 | grep -qF "$object_url"; then
echo "OK — Objecten published to NRC and the abonnement delivered it:"
docker logs rr-osink 2>&1 | grep -F "$object_url" | tail -1 | cut -c1-500
exit 0
fi
sleep 2
done
echo "FAIL — no 'objecten' notification for $object_url reached the sink." >&2
echo " Objecten accepted the write, so the gap is downstream: the celery broker/worker," >&2
echo " the kanaal registration, or Objecten's notifications_config." >&2
echo "--- sink log ---" >&2; docker logs rr-osink 2>&1 | tail -8 >&2
echo "--- objecten log ---" >&2; docker logs "$obj" 2>&1 | tail -15 >&2
exit 1