feat: read projection sourced from the register in Objecten (closes #153) (#155)

## What & why S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the `RegisterRecord` in Objecten, not from ZGW zaak events.** Until now the subscriber listened on `zaken` and *inferred* register state from case events — a `zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification at all, so every projection made a second hop to the ACL. The register — a fact about a person — was being reconstructed by guessing at the lifecycle of the case that produced it. - The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish). - An Objecten notification carries **no record data**, only the object URL, so the record is read back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as ADR-0028 established. - The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`, `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all gone, and so is the ACL enrichment hop. - **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently drop every submitted registration from the public register, since only approval wrote a record. - `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all. **ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must agree" — is closed: there is one source now. Closes #153 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing tests committed before the implementation — two red/green pairs, ACL side (06c0444 → 566ef7d) and subscriber side (142ed45 → 8af09b2). - [x] Refactor commit follows (b496ac9). - [x] Conventional Commits referencing the issue (`refs #153`). - [x] CI green — all six jobs on b30fa66, `verify-stack` end to end including the e2e. - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes (`verify-stack`'s bring-up step — see the wait-healthy fix below). - [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md, e2e header comment. - [x] ADR added in `docs/architecture/`. - [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register shows the same two statuses for the same registrations; only where they come from changed. ## Notes for reviewers **The decision I'd most like a second opinion on** is the one the issue didn't settle: what happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice between (a) the ACL also writing on submit, (b) a public register that lists only actual registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling ADR-0028 set out to remove. All three are laid out in ADR-0030. **The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the object URL (the ACL upserts *one object per registration*, so submit and approval notify about the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6. **The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` → `register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning something else, and a rebuild would then have projected that garbage. It also empties both tables: a pre-slice row describes a zaak event the new projector can't reproject, and those registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten if a long-lived environment ever needs it. **`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back produces no projection row — that's the re-source working, not a gap. ## Three fixes CI found, none of them in the projection logic 1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db` the same way. Anchored on the compose replica suffix, which the verify scripts already do. 2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL put this check in the same bind run-domain-check.sh already handles: `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak reflects the request Host into the zaaktype URL and then rejects it on zaak-create when single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`. 3. **Approval arrives as `partial_update`, not `update`** (0dd26a7 → b30fa66) — the one real bug in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but names the action `partial_update`, so the projector dropped every approval. Only the e2e could catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that drives a *real* approval. `verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` / `distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace chain being broken. Filed as **#156** rather than absorbed here. **Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin image, so `objecten-celery` is already a single-process worker. Noted in #156. **Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring `objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too broad a ripple to land inside an unrelated slice — worth its own issue. **Known costs, all in the ADR:** submission is now two writes across two modules and eventually consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc` asserts that path.Reviewed-on: #155
2026-09-01 07:26:33 +00:00
parent 2125fb0cfd
commit 94742a261f
30 changed files with 821 additions and 251 deletions
@@ -90,6 +90,16 @@ app.MapPost("/zaken/reference", async (ZaakReferenceRequest body, AclService acl
    return Results.Ok(new { reference });
 });

+// Read the register record an object in Objecten holds. The Event Subscriber projects a register
+// write from the notification NRC delivers, which carries only the object URL, and may not talk to
+// Objecten itself (§8.1, ADR-0028/ADR-0030). 404 when the object holds no record — the subscriber
+// treats that as "nothing to project" rather than an error (§8.6).
+app.MapPost("/register-records/read", async (RegisterRecordReadRequest body, AclService acl, CancellationToken ct) =>
+{
+    var record = await acl.GetRegisterRecordAsync(new Uri(body.ObjectUrl), ct);
+    return record is null ? Results.NotFound() : Results.Ok(record);
+});
+
 // Store an uploaded diploma against a zaak (S-10b): the domain sends the file as base64; the ACL
 // creates the ZGW enkelvoudiginformatieobject and relates it to the zaak (§8.1). Returns its URL.
 app.MapPost("/documenten", async (StoreDocumentRequest body, AclService acl, CancellationToken ct) =>
@@ -131,6 +141,9 @@ public sealed record CancelZaakRequest(string ZaakUrl);

 public sealed record ZaakReferenceRequest(string ZaakUrl);

+/// <summary>The object whose register record the Event Subscriber wants read back (S-19b-2).</summary>
+public sealed record RegisterRecordReadRequest(string ObjectUrl);
+
 public sealed record StoreDocumentRequest(string ZaakUrl, string ContentBase64, string FileName, string ContentType);

 public partial class Program;
@@ -24,7 +24,16 @@ public sealed class AclService(
            clock.Today,
            registration.Reference);

-        return await gateway.OpenZaakAsync(request, ct);
+        var zaakUrl = await gateway.OpenZaakAsync(request, ct);
+
+        // The register — not ZGW — is what the read projection is sourced from (ADR-0028/ADR-0030),
+        // so the record exists from submission, not only from approval. Same two-writes-converging
+        // posture as ApproveZaakAsync: the upsert is keyed on the zaak id, so a retried submit
+        // updates the record rather than adding a second one (§8.6).
+        await register.UpsertAsync(
+            new RegisterRecord(ZaakId(zaakUrl), RegisterRecordStatus.Ingediend, registration.Reference), ct);
+
+        return zaakUrl;
    }

    /// <summary>
@@ -52,6 +61,18 @@ public sealed class AclService(
            ct);
    }

+    /// <summary>
+    /// The register record held by an object in Objecten, for the Event Subscriber (S-19b-2). The
+    /// subscriber gets only an object URL on the notification and may not read Objecten itself
+    /// (§8.1, ADR-0028), so the ACL reads it back.
+    /// </summary>
+    public Task<RegisterRecord?> GetRegisterRecordAsync(Uri objectUrl, CancellationToken ct = default)
+    {
+        ArgumentNullException.ThrowIfNull(objectUrl);
+
+        return register.GetAsync(objectUrl, ct);
+    }
+
    /// <summary>The zaak's UUID — the key the register record and the read projection rows share.</summary>
    private static string ZaakId(Uri zaakUrl) => zaakUrl.Segments[^1].TrimEnd('/');

@@ -13,6 +13,14 @@ public interface IRegisterRecordGateway
    /// the existing object instead of creating a second one (§8.6).
    /// </summary>
    Task UpsertAsync(RegisterRecord record, CancellationToken ct = default);
+
+    /// <summary>
+    /// The register record held by the object at <paramref name="objectUrl"/>, or <c>null</c> if that
+    /// object holds none. The Event Subscriber projects a register write from the notification NRC
+    /// delivers, which carries only the object URL — so it reads the record back through the ACL
+    /// rather than talking to Objecten itself (§8.1, S-19b-2).
+    /// </summary>
+    Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default);
 }

 /// <summary>
@@ -1,3 +1,4 @@
+using System.Net;
 using System.Net.Http.Headers;
 using System.Net.Http.Json;
 using System.Text.Json.Serialization;
@@ -38,6 +39,30 @@ public sealed class ObjectenGateway(HttpClient http, ObjectenOptions options, IC
                "Updating the register record", ct);
    }

+    public async Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default)
+    {
+        ArgumentNullException.ThrowIfNull(objectUrl);
+
+        // Fetched by the URL the notification carried, so no objecttype resolution and no search —
+        // unlike a write, which has to find the object for a registration id.
+        using var message = new HttpRequestMessage(HttpMethod.Get, objectUrl);
+        message.Headers.Authorization = new AuthenticationHeaderValue("Token", options.Token);
+        message.Headers.Add("Accept-Crs", "EPSG:4326");
+
+        using var response = await http.SendAsync(message, ct);
+        // The object may be gone by the time a (possibly redelivered) notification is handled —
+        // there is simply nothing to project, which is not a failure (§8.6).
+        if (response.StatusCode == HttpStatusCode.NotFound)
+            return null;
+
+        await EnsureSuccessAsync(response, "Reading the register record", ct);
+
+        var body = await response.Content.ReadFromJsonAsync<ReadObjectDto>(ct)
+            ?? throw new InvalidOperationException("Objecten returned an empty object response");
+        var data = body.Record?.Data;
+        return data is null ? null : new RegisterRecord(data.Id, data.Status, data.Reference);
+    }
+
    private RecordDto NewRecord(int typeVersion, RecordDataDto data) =>
        new(typeVersion, data, clock.Today.ToString("yyyy-MM-dd"));

@@ -141,6 +166,12 @@ public sealed class ObjectenGateway(HttpClient http, ObjectenOptions options, IC
    private sealed record ObjectDto(
        [property: JsonPropertyName("url")] string Url);

+    private sealed record ReadObjectDto(
+        [property: JsonPropertyName("record")] ReadRecordDto? Record);
+
+    private sealed record ReadRecordDto(
+        [property: JsonPropertyName("data")] RecordDataDto? Data);
+
    private sealed record CreateObjectDto(
        [property: JsonPropertyName("type")] string Type,
        [property: JsonPropertyName("record")] RecordDto Record);
@@ -79,11 +79,21 @@ public class AclServiceTests
    {
        public readonly List<RegisterRecord> Upserted = [];

+        public RegisterRecord? Stored;
+
+        public Uri? ReadFrom;
+
        public Task UpsertAsync(RegisterRecord record, CancellationToken ct = default)
        {
            Upserted.Add(record);
            return Task.CompletedTask;
        }
+
+        public Task<RegisterRecord?> GetAsync(Uri objectUrl, CancellationToken ct = default)
+        {
+            ReadFrom = objectUrl;
+            return Task.FromResult(Stored);
+        }
    }

    private static AclDefaults Defaults() => new()
@@ -130,6 +140,52 @@ public class AclServiceTests
        Assert.Equal("reg-77", req.Identificatie);
    }

+    [Fact]
+    public async Task Opening_a_zaak_also_writes_an_ingediend_register_record(/* S-19b-2 */)
+    {
+        var gateway = new FakeGateway();
+        var register = new FakeRegisterRecordGateway();
+        var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
+
+        await service.OpenZaakAsync(new DomainRegistration("123456782", "reg-77"));
+
+        // The register — not ZGW — is what the read projection is sourced from (ADR-0028), so a
+        // submitted registration has to exist there the moment the zaak is opened, not only on
+        // approval. Approval upserts this same record to INGESCHREVEN.
+        var record = Assert.Single(register.Upserted);
+        Assert.Equal("abc", record.Id);
+        Assert.Equal("INGEDIEND", record.Status);
+        // The reference comes from the registration itself — no ZGW read-back needed on this path.
+        Assert.Equal("reg-77", record.Reference);
+    }
+
+    [Fact]
+    public async Task Reading_a_register_record_goes_through_the_objecten_gateway(/* S-19b-2 */)
+    {
+        var gateway = new FakeGateway();
+        var register = new FakeRegisterRecordGateway { Stored = new RegisterRecord("abc", "INGESCHREVEN", "reg-77") };
+        var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
+        var objectUrl = new Uri("http://objecten.local:8000/api/v2/objects/9de4a2ca");
+
+        var record = await service.GetRegisterRecordAsync(objectUrl);
+
+        Assert.Equal(objectUrl, register.ReadFrom);
+        Assert.Equal("abc", record!.Id);
+        Assert.Equal("INGESCHREVEN", record.Status);
+        Assert.Equal("reg-77", record.Reference);
+    }
+
+    [Fact]
+    public async Task Reading_a_register_record_from_a_null_url_is_rejected(/* S-19b-2 */)
+    {
+        var gateway = new FakeGateway();
+        var register = new FakeRegisterRecordGateway();
+        var service = ServiceWith(gateway, register, Defaults(), new DateOnly(2026, 6, 4));
+
+        await Assert.ThrowsAsync<ArgumentNullException>(() => service.GetRegisterRecordAsync(null!));
+        Assert.Null(register.ReadFrom);
+    }
+
    [Fact]
    public async Task Opening_a_zaak_reflects_a_default_fill_update(/* S-15b */)
    {
@@ -83,6 +83,43 @@ public class ObjectenGatewayTests

    private static RegisterRecord Record() => new("zaak-uuid-1", RegisterRecordStatus.Ingeschreven, "REG-2026-0001");

+    [Fact]
+    public async Task Reads_a_register_record_back_from_its_object_url(/* S-19b-2 */)
+    {
+        var sent = new List<Sent>();
+        var objectUrl = new Uri("http://objecten:8000/api/v2/objects/obj-9");
+        var gateway = Gateway(sent, _ => Json(new
+        {
+            url = objectUrl.ToString(),
+            record = new { data = new { id = "zaak-uuid-1", status = "INGESCHREVEN", reference = "REG-2026-0001" } },
+        }));
+
+        var record = await gateway.GetAsync(objectUrl);
+
+        // The object is fetched directly by the URL the notification carried — no objecttype
+        // resolution and no search, unlike a write.
+        var read = Assert.Single(sent);
+        Assert.Equal(HttpMethod.Get, read.Method);
+        Assert.Equal(objectUrl, read.Uri);
+        // Objecten is a geo API: the CRS header is required on reads too.
+        Assert.Equal("EPSG:4326", read.AcceptCrs);
+        Assert.Equal("Token objecten-token", read.Auth);
+        Assert.Equal("zaak-uuid-1", record!.Id);
+        Assert.Equal("INGESCHREVEN", record.Status);
+        Assert.Equal("REG-2026-0001", record.Reference);
+    }
+
+    [Fact]
+    public async Task Reading_an_object_that_is_gone_yields_no_record(/* S-19b-2 */)
+    {
+        var sent = new List<Sent>();
+        var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.NotFound));
+
+        // A record deleted between the notification and the read is not an error — there is simply
+        // nothing to project (§8.6: the subscriber tolerates whatever order deliveries arrive in).
+        Assert.Null(await gateway.GetAsync(new Uri("http://objecten:8000/api/v2/objects/gone")));
+    }
+
    [Fact]
    public async Task Creates_the_object_when_none_exists_for_the_registration()
    {