Files
register-referentie/services/acl/Acl.Tests/ObjectenGatewayTests.cs
T
not 94742a261f
CI / build (push) Successful in 1m7s
CI / lint (push) Successful in 1m26s
CI / unit (push) Successful in 1m37s
CI / frontend (push) Successful in 3m36s
CI / mutation (push) Successful in 6m42s
CI / verify-stack (push) Failing after 11m26s
feat: read projection sourced from the register in Objecten (closes #153) (#155)
## What & why

S-19b-2, closing out ADR-0028's stated direction: **the read projection is now derived from the
`RegisterRecord` in Objecten, not from ZGW zaak events.**

Until now the subscriber listened on `zaken` and *inferred* register state from case events — a
`zaak/create` meant INGEDIEND, and any `status/create` was assumed to be the approval (it may not
read OpenZaak, so it could not tell statustypen apart). The reference wasn't in the notification
at all, so every projection made a second hop to the ACL. The register — a fact about a person —
was being reconstructed by guessing at the lifecycle of the case that produced it.

- The subscriber's abonnement moves to the `objecten` kanaal (S-19b-1 made it publish).
- An Objecten notification carries **no record data**, only the object URL, so the record is read
  back through the ACL (`POST /register-records/read`) — §8.1 applies to Objecten exactly as
  ADR-0028 established.
- The record carries `id`, `status` and `reference`, so the row *is* the record: `IsZaakCreated`,
  `IsZaakStatusSet`, `ZaakUrl`, `ZaakId` and `ToEntry`'s `Resource == "status"` inference are all
  gone, and so is the ACL enrichment hop.
- **The ACL now writes an INGEDIEND record on submit.** Without it, re-sourcing would silently
  drop every submitted registration from the public register, since only approval wrote a record.
- `processed_notifications` holds the projected row (`register_id`, `status`, `reference`) instead
  of the ZGW event, so a rebuild is a replay with no mapping rules and no upstream reads at all.

**ADR-0030** records it. ADR-0028's open caveat — record written but not yet read, "the two must
agree" — is closed: there is one source now.

Closes #153

## Definition of Done

- [x] Linked Gitea issue (above).
- [x] Failing tests committed before the implementation — two red/green pairs, ACL side
      (06c0444566ef7d) and subscriber side (142ed458af09b2).
- [x] Refactor commit follows (b496ac9).
- [x] Conventional Commits referencing the issue (`refs #153`).
- [x] CI green — all six jobs on b30fa66, `verify-stack` end to end including the e2e.
- [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes
      (`verify-stack`'s bring-up step — see the wait-healthy fix below).
- [x] Docs updated — ADR-0030 added, ADR-0028's consequence + caveat annotated, BACKLOG.md,
      e2e header comment.
- [x] ADR added in `docs/architecture/`.
- [x] Demo note in `docs/demo-script.md` — n/a: no user-visible change. The openbaar register
      shows the same two statuses for the same registrations; only where they come from changed.

## Notes for reviewers

**The decision I'd most like a second opinion on** is the one the issue didn't settle: what
happens to INGEDIEND. Objecten held only INGESCHREVEN records, so re-sourcing forced a choice
between (a) the ACL also writing on submit, (b) a public register that lists only actual
registrations, or (c) a hybrid keeping both kanalen. I took (a): visible behaviour is unchanged
and the register holds the whole lifecycle. (b) is arguably the better *semantics* for a public
register but narrows what the portal shows and reads against PRD §68 ("~50 register entries with
diverse statuses"); (c) leaves the projection half-derived from ZGW, which is the coupling
ADR-0028 set out to remove. All three are laid out in ADR-0030.

**The dedup key is the projected row**, `objecten:object:{url}:{status}:{reference}` — not the
object URL (the ACL upserts *one object per registration*, so submit and approval notify about
the same URL and the approval would be swallowed as a duplicate) and not URL+actie (a retried
approval is a second `update`). Redeliveries collapse, genuine state changes don't. §8.6.

**The migration drops columns rather than renaming them.** EF scaffolded renames — `resource` →
`register_id`, `zaak_id` → `status` — which would have carried ZGW values into columns meaning
something else, and a rebuild would then have projected that garbage. It also empties both
tables: a pre-slice row describes a zaak event the new projector can't reproject, and those
registrations have no RegisterRecord in Objecten either, so they're not re-derivable from the new
source. Stated as a ceiling in the ADR — fine while stacks are ephemeral, backfill from Objecten
if a long-lived environment ever needs it.

**`run-projection-check.sh` now opens its zaak through the ACL** instead of straight against
OpenZaak, because the ACL is what writes the record. A zaak created behind the ACL's back
produces no projection row — that's the re-source working, not a gap.

## Three fixes CI found, none of them in the projection logic

1. **`wait-healthy.sh` matched the wrong container** (744f91a). Bring-up timed out with
   `TIMEOUT: 'objecten' not healthy (status=none)` while the `docker ps` it dumps showed
   objecten `Up 9 minutes (healthy)`. `--filter name=` is a substring match, so `objecten` also
   matches `objecten-db`/`objecten-redis`/`objecten-celery`, and `head -1` took whichever docker
   listed first — the celery worker has no healthcheck, hence `status=none`. Latent since those
   services landed and decided purely by listing order; `objecttypen` matches `objecttypen-db`
   the same way. Anchored on the compose replica suffix, which the verify scripts already do.
2. **The ACL had to be repointed at OpenZaak's IP** (7e0897a). Opening the zaak through the ACL
   put this check in the same bind run-domain-check.sh already handles:
   `400 {"name":"zaaktype","code":"bad-url","reason":"Voer een geldige URL in."}`. OpenZaak
   reflects the request Host into the zaaktype URL and then rejects it on zaak-create when
   single-label — the mechanism compose already documents on `ACL_OPENZAAK_BASEURL`.
3. **Approval arrives as `partial_update`, not `update`** (0dd26a7b30fa66) — the one real bug
   in the slice. The ACL upserts with PATCH; DRF routes it through the notifying `update()` but
   names the action `partial_update`, so the projector dropped every approval. Only the e2e could
   catch it: `verify-projection` drives a submit, and per ADR-0028 the e2e is the only check that
   drives a *real* approval.

`verify-tracing` also failed once (run 722) on a path this PR doesn't touch, and passed on a
plain re-run of the same commit. Tempo logged `pusher failed to consume trace data` /
`distributor_pool failing healthcheck` — it dropped spans under runner load rather than the trace
chain being broken. Filed as **#156** rather than absorbed here.

**Correction to the #152 PR notes:** I wrote there that celery concurrency was "the next knob" if
verify-stack got tight. It isn't — `CELERY_WORKER_CONCURRENCY` already defaults to 1 in the Maykin
image, so `objecten-celery` is already a single-process worker. Noted in #156.

**Possible follow-up, deliberately not done here:** an `openzaak.local` network alias mirroring
`objecten.local` would remove the ACL-repoint dance from both run-domain-check.sh and
run-projection-check.sh. It changes the host in every zaak URL the system produces, which is too
broad a ripple to land inside an unrelated slice — worth its own issue.

**Known costs, all in the ADR:** submission is now two writes across two modules and eventually
consistent (same posture ADR-0028 accepted for approval); projecting now depends on the ACL being
reachable on the main path, not just for enrichment (NRC retries, so it converges); and OpenZaak
still publishes to `zaken` with nothing in the product listening — kept because `verify-nrc`
asserts that path.Reviewed-on: #155
2026-09-01 07:26:33 +00:00

368 lines
16 KiB
C#

using System.Net;
using System.Net.Http.Json;
using Acl.Application;
using Acl.Infrastructure;
namespace Acl.Tests;
public class ObjectenGatewayTests
{
private sealed class StubHandler(Func<HttpRequestMessage, Task<HttpResponseMessage>> onSend)
: HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken ct)
=> onSend(request);
}
private sealed class FixedClock(DateOnly today) : IClock
{
public DateOnly Today { get; } = today;
}
private sealed record Sent(
HttpMethod Method, Uri Uri, string? Body, string? Auth, string? ContentCrs, string? AcceptCrs, long? ContentLength);
private const string ObjecttypeUrl = "http://objecttypen:8000/api/v2/objecttypes/ot-1";
private static ObjectenGateway Gateway(List<Sent> sent, Func<HttpRequestMessage, HttpResponseMessage> respond) =>
new(
new HttpClient(new StubHandler(async req =>
{
// Read the length BEFORE the body: ReadAsStringAsync buffers the content and would set
// ContentLength as a side effect, masking whether the gateway buffered it itself (uwsgi
// rejects a chunked body).
sent.Add(new Sent(
req.Method,
req.RequestUri!,
ContentLength: req.Content?.Headers.ContentLength,
Body: req.Content is null ? null : await req.Content.ReadAsStringAsync(),
Auth: req.Headers.Authorization?.ToString(),
ContentCrs: req.Content?.Headers.TryGetValues("Content-Crs", out var c) == true ? string.Join(",", c!) : null,
AcceptCrs: req.Headers.TryGetValues("Accept-Crs", out var a) ? string.Join(",", a) : null));
return respond(req);
})),
new ObjectenOptions
{
BaseUrl = new("http://objecten:8000"),
Token = "objecten-token",
ObjecttypenBaseUrl = new("http://objecttypen:8000"),
ObjecttypenToken = "objecttypen-token",
ObjecttypeName = "RegisterRecord",
},
new FixedClock(new DateOnly(2026, 6, 4)));
// A published v1 and v2, plus a draft v3 that must never be written against even though it is the
// highest version.
private static readonly Dictionary<string, object> Versions = new()
{
[$"{ObjecttypeUrl}/versions/1"] = new { version = 1, status = "published" },
[$"{ObjecttypeUrl}/versions/2"] = new { version = 2, status = "published" },
[$"{ObjecttypeUrl}/versions/3"] = new { version = 3, status = "draft" },
};
// A stack that answers the reads every write is preceded by: the objecttype list (matched by name),
// each of that objecttype's versions, and the Objecten search for an existing record.
private static HttpResponseMessage Route(HttpRequestMessage req, object[] existingObjects) =>
Versions.TryGetValue(req.RequestUri!.ToString(), out var version)
? Json(version)
: req.RequestUri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)
? Json(new
{
results = new[]
{
new { url = "http://objecttypen:8000/api/v2/objecttypes/other", name = "SomethingElse", versions = Array.Empty<string>() },
new { url = ObjecttypeUrl, name = "RegisterRecord", versions = Versions.Keys.ToArray() },
},
})
: req.Method == HttpMethod.Get
? Json(new { results = existingObjects })
: new HttpResponseMessage(HttpStatusCode.Created) { Content = JsonContent.Create(new { url = "http://objecten:8000/api/v2/objects/obj-1" }) };
private static HttpResponseMessage Json(object body) =>
new(HttpStatusCode.OK) { Content = JsonContent.Create(body) };
private static RegisterRecord Record() => new("zaak-uuid-1", RegisterRecordStatus.Ingeschreven, "REG-2026-0001");
[Fact]
public async Task Reads_a_register_record_back_from_its_object_url(/* S-19b-2 */)
{
var sent = new List<Sent>();
var objectUrl = new Uri("http://objecten:8000/api/v2/objects/obj-9");
var gateway = Gateway(sent, _ => Json(new
{
url = objectUrl.ToString(),
record = new { data = new { id = "zaak-uuid-1", status = "INGESCHREVEN", reference = "REG-2026-0001" } },
}));
var record = await gateway.GetAsync(objectUrl);
// The object is fetched directly by the URL the notification carried — no objecttype
// resolution and no search, unlike a write.
var read = Assert.Single(sent);
Assert.Equal(HttpMethod.Get, read.Method);
Assert.Equal(objectUrl, read.Uri);
// Objecten is a geo API: the CRS header is required on reads too.
Assert.Equal("EPSG:4326", read.AcceptCrs);
Assert.Equal("Token objecten-token", read.Auth);
Assert.Equal("zaak-uuid-1", record!.Id);
Assert.Equal("INGESCHREVEN", record.Status);
Assert.Equal("REG-2026-0001", record.Reference);
}
[Fact]
public async Task Reading_an_object_that_is_gone_yields_no_record(/* S-19b-2 */)
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.NotFound));
// A record deleted between the notification and the read is not an error — there is simply
// nothing to project (§8.6: the subscriber tolerates whatever order deliveries arrive in).
Assert.Null(await gateway.GetAsync(new Uri("http://objecten:8000/api/v2/objects/gone")));
}
[Fact]
public async Task Creates_the_object_when_none_exists_for_the_registration()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var write = sent.Single(s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.Contains($"\"type\":\"{ObjecttypeUrl}\"", write.Body);
// The highest *published* version (2), not the highest version (a draft 3).
Assert.Contains("\"typeVersion\":2", write.Body);
Assert.Contains("\"id\":\"zaak-uuid-1\"", write.Body);
Assert.Contains("\"status\":\"INGESCHREVEN\"", write.Body);
Assert.Contains("\"reference\":\"REG-2026-0001\"", write.Body);
Assert.Contains("\"startAt\":\"2026-06-04\"", write.Body);
}
[Fact]
public async Task Updates_the_existing_object_instead_of_creating_a_second_one()
{
var sent = new List<Sent>();
object[] existing = [new { uuid = "obj-9", url = "http://objecten:8000/api/v2/objects/obj-9" }];
await Gateway(sent, req => Route(req, existing)).UpsertAsync(Record());
Assert.DoesNotContain(sent, s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
var write = sent.Single(s => s.Method == HttpMethod.Patch);
Assert.Equal("http://objecten:8000/api/v2/objects/obj-9", write.Uri.ToString());
Assert.Contains("\"status\":\"INGESCHREVEN\"", write.Body);
}
[Fact]
public async Task Searches_objecten_for_the_registration_id_within_the_objecttype()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var search = sent.Single(s => s.Method == HttpMethod.Get && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.Contains("type=" + Uri.EscapeDataString(ObjecttypeUrl), search.Uri.Query);
Assert.Contains("data_attrs=id__exact__zaak-uuid-1", search.Uri.Query);
}
[Fact]
public async Task Authenticates_with_the_static_token_of_each_api()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)),
s => Assert.Equal("Token objecttypen-token", s.Auth));
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objects", StringComparison.Ordinal)),
s => Assert.Equal("Token objecten-token", s.Auth));
}
[Fact]
public async Task Sends_the_geo_crs_headers_the_objecten_api_requires()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var objects = sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objects", StringComparison.Ordinal)).ToList();
Assert.All(objects, s => Assert.Equal("EPSG:4326", s.AcceptCrs));
Assert.All(objects.Where(s => s.Body is not null), s => Assert.Equal("EPSG:4326", s.ContentCrs));
}
[Fact]
public async Task Resolves_the_objecttype_once_and_reuses_it_across_writes()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => Route(req, []));
await gateway.UpsertAsync(Record());
await gateway.UpsertAsync(Record() with { Id = "zaak-uuid-2" });
Assert.Single(sent, s => s.Uri.AbsolutePath == "/api/v2/objecttypes");
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_has_no_published_version()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.RequestUri!.AbsolutePath.Contains("/versions/", StringComparison.Ordinal)
? Json(new { version = 1, status = "draft" })
: Route(req, []));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("published version", error.Message);
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_is_not_registered()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = JsonContent.Create(new { results = Array.Empty<object>() }),
});
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("RegisterRecord", error.Message);
}
[Fact]
public async Task Surfaces_the_objecten_error_body_when_a_write_is_rejected()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Post && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? new HttpResponseMessage(HttpStatusCode.BadRequest) { Content = new StringContent("{\"detail\":\"schema mismatch\"}") }
: Route(req, []));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("schema mismatch", error.Message);
Assert.Contains("Creating the register record", error.Message);
}
[Fact]
public async Task Surfaces_the_objecten_error_body_when_an_update_is_rejected()
{
var sent = new List<Sent>();
object[] existing = [new { url = "http://objecten:8000/api/v2/objects/obj-9" }];
var gateway = Gateway(sent, req => req.Method == HttpMethod.Patch
? new HttpResponseMessage(HttpStatusCode.BadRequest) { Content = new StringContent("{\"detail\":\"stale version\"}") }
: Route(req, existing));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("stale version", error.Message);
Assert.Contains("Updating the register record", error.Message);
}
[Fact]
public async Task Surfaces_a_failed_read_instead_of_writing_blind()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.Unauthorized)
{
Content = new StringContent("{\"detail\":\"invalid token\"}"),
});
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("Querying objecttypen", error.Message);
Assert.Contains("invalid token", error.Message);
// A read that failed must never be mistaken for "nothing there yet" and followed by a write.
Assert.DoesNotContain(sent, s => s.Method == HttpMethod.Post || s.Method == HttpMethod.Patch);
}
[Fact]
public async Task Fails_loudly_when_the_objecttype_carries_no_versions_at_all()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.RequestUri!.AbsolutePath == "/api/v2/objecttypes"
? Json(new { results = new[] { new { url = ObjecttypeUrl, name = "RegisterRecord" } } })
: Route(req, []));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("published version", error.Message);
}
[Fact]
public async Task Says_which_read_failed_when_the_objecten_search_errors()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Get && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? new HttpResponseMessage(HttpStatusCode.InternalServerError) { Content = new StringContent("boom") }
: Route(req, []));
var error = await Assert.ThrowsAsync<HttpRequestException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("Querying objects", error.Message);
}
[Fact]
public async Task Surfaces_an_empty_read_body_rather_than_dereferencing_it()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, _ => new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("null", System.Text.Encoding.UTF8, "application/json"),
});
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("objecttypen", error.Message);
}
[Fact]
public async Task Treats_a_result_less_response_as_no_match_rather_than_crashing()
{
var sent = new List<Sent>();
// The objecttypes collection carries no `results` key — the objecttype is absent, which must
// surface as the "not registered" error rather than an ArgumentNullException from LINQ.
var gateway = Gateway(sent, _ => Json(new { }));
var error = await Assert.ThrowsAsync<InvalidOperationException>(() => gateway.UpsertAsync(Record()));
Assert.Contains("RegisterRecord", error.Message);
}
[Fact]
public async Task Creates_the_object_when_the_search_response_carries_no_results_key()
{
var sent = new List<Sent>();
var gateway = Gateway(sent, req => req.Method == HttpMethod.Get && req.RequestUri!.AbsolutePath == "/api/v2/objects"
? Json(new { })
: Route(req, []));
await gateway.UpsertAsync(Record());
Assert.Contains(sent, s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
}
[Fact]
public async Task Reads_objecttypen_without_the_crs_headers_it_does_not_accept()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
// Objecttypen is not a geo API; only the Objecten hops carry CRS.
Assert.All(
sent.Where(s => s.Uri.AbsolutePath.StartsWith("/api/v2/objecttypes", StringComparison.Ordinal)),
s => Assert.Null(s.AcceptCrs));
}
[Fact]
public async Task Buffers_the_write_body_so_uwsgi_gets_a_content_length()
{
var sent = new List<Sent>();
await Gateway(sent, req => Route(req, [])).UpsertAsync(Record());
var write = sent.Single(s => s.Method == HttpMethod.Post && s.Uri.AbsolutePath == "/api/v2/objects");
Assert.NotNull(write.ContentLength);
}
[Fact]
public async Task Rejects_a_null_record_without_calling_objecten()
{
var sent = new List<Sent>();
await Assert.ThrowsAsync<ArgumentNullException>(() => Gateway(sent, req => Route(req, [])).UpsertAsync(null!));
Assert.Empty(sent);
}
}