Files
not d5dfbdc0b2
CI / build (push) Successful in 1m47s
CI / lint (push) Successful in 1m59s
CI / unit (push) Successful in 1m57s
CI / frontend (push) Successful in 3m58s
CI / mutation (push) Successful in 6m57s
CI / verify-stack (push) Successful in 8m29s
feat(obs): Prometheus metrics on /metrics + golden-signal Grafana dashboard (closes #124) (#129)
## What & why

S-16c, the last of the S-16 (#17) split, on top of the backplane (#122) and distributed tracing (#123). The five .NET services now expose OpenTelemetry **metrics** in Prometheus format at `/metrics`; Prometheus scrapes each (one job per service); and Grafana ships a pre-built **Request path — golden signals** dashboard (traffic / errors / latency / saturation), split by service.

Closes #124

### How

- Each service adds `.WithMetrics(AddAspNetCoreInstrumentation + AddHttpClientInstrumentation + AddMeter("System.Runtime") + AddPrometheusExporter)` and maps `/metrics`. Same shape as the S-16b tracing wiring already in these `Program.cs` files.
- `infra/observability/prometheus/prometheus.yml`: one scrape job per service (`acl`, `domain`, `bff`, `event-subscriber`, `projection-api`), reached by compose service name.
- `infra/observability/grafana/provisioning/dashboards/`: dashboard provider + `golden-signals.json` (baked into the Grafana image by the existing `COPY provisioning/`).
- `verify-metrics` (new CI verify-stack step + Makefile target): generates BFF traffic and asserts Prometheus scraped the golden-signal metric from every service. Mirrors `verify-tracing`.

### Dependency (CLAUDE.md §13/§14)

Adds `OpenTelemetry.Exporter.Prometheus.AspNetCore` `1.17.0-beta.1` (matched to the `1.17.0` core already in use). It gives the OTel-native `/metrics` pull endpoint; replacing it would mean hand-rolling Prometheus exposition over a `MeterListener`; the risk is that it is a **prerelease** package (the whole OTel .NET Prometheus line is `-beta`) — pinned, wired only in `Program.cs`, and gated by `verify-metrics`. Recorded in **ADR-0024**.

## Definition of Done

- [x] Linked Gitea issue (#124).
- [x] Failing test committed before the implementation (`test(bff): /metrics exposes http-server request duration`).
- [x] Implementation makes the test pass.
- [ ] CI green — pending Gitea Actions run.
- [x] `docker compose up` reaches green health within 3 min (backplane images unchanged in shape; not on the health gate, ADR-0023).
- [x] Docs updated — demo-script S-16c entry.
- [x] ADR added — ADR-0024.
- [x] Demo note in `docs/demo-script.md`.

## Notes for reviewers

- `/health` polls are counted as traffic (metrics aren't path-filtered, unlike traces). Fine for a demo dashboard and honest — real load stacks on top.
- `projection-api` has no Stryker config (unchanged); the four mutated services carry the metrics wiring in `Program.cs`, same as the merged S-16b tracing code.
- Metric names verified against a live service: `http_server_request_duration_seconds{,_bucket,_count}`, label `http_response_status_code`, `dotnet_process_cpu_time_seconds_total`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #129
2026-07-24 08:31:41 +00:00

99 lines
4.6 KiB
C#

using System.Text.Json;
using EventSubscriber.Application;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using Projection.ReadModel;
var builder = WebApplication.CreateBuilder(args);
// OpenTelemetry tracing (S-16b, ADR-0023): auto-instrument the incoming NRC notification callback and
// the outgoing ACL enrichment call, exported over OTLP to Tempo. Service name + OTLP endpoint come
// from OTEL_* env (compose); the exporter no-ops when Tempo is unreachable.
builder.Services.AddOpenTelemetry()
.ConfigureResource(r => r.AddService(
builder.Configuration["OTEL_SERVICE_NAME"] ?? builder.Environment.ApplicationName))
.WithTracing(tracing => tracing
.AddAspNetCoreInstrumentation(o => o.Filter = ctx => ctx.Request.Path != "/health")
.AddHttpClientInstrumentation()
.AddOtlpExporter())
// OpenTelemetry metrics (S-16c, ADR-0023): golden signals for the request path —
// http.server.request.duration (traffic/errors/latency) + http.client.* for downstream hops, plus
// the built-in System.Runtime meter for saturation (GC, CPU, thread pool). Prometheus scrapes these
// from /metrics (mapped below); metrics aren't pushed over OTLP, so no collector hop (ADR-0023).
.WithMetrics(metrics => metrics
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddMeter("System.Runtime")
.AddPrometheusExporter());
var connectionString = builder.Configuration.GetConnectionString("Projection")
?? throw new InvalidOperationException("Missing connection string 'ConnectionStrings:Projection'");
// The exact Authorization header value Open Notificaties sends on each abonnement callback.
// NRC probes the callback during registration and refuses it unless it returns 401 without
// this value (ADR-0007), so the webhook enforces it.
var webhookToken = builder.Configuration["EventSubscriber:Webhook:AuthToken"]
?? throw new InvalidOperationException("Missing configuration 'EventSubscriber:Webhook:AuthToken'");
// The ACL is the only code that may read ZGW (§8.1); the subscriber enriches the projection with the
// zaak reference through it (adr-proposal #78).
var aclBaseUrl = builder.Configuration["Acl:BaseUrl"]
?? throw new InvalidOperationException("Missing configuration 'Acl:BaseUrl'");
builder.Services.AddProjectionReadModel(connectionString);
builder.Services.AddProjectionWriteSide();
builder.Services.AddHttpClient<EventSubscriber.Application.IAclClient, EventSubscriber.Api.AclHttpClient>(
c => c.BaseAddress = new Uri(aclBaseUrl));
var app = builder.Build();
// Apply migrations on start so a fresh stack reaches a usable schema unattended.
await app.Services.MigrateProjectionAsync();
app.MapGet("/health", () => "Healthy");
// Prometheus scrape endpoint (S-16c): exposes the OTel metrics above in Prometheus text format.
app.MapPrometheusScrapingEndpoint();
// The NRC abonnement callback. Open Notificaties POSTs a notification here; we project it.
// Auth-on-callback is mandatory: the auth check runs *before* the body is read, so NRC's
// registration probe (a POST without the configured Authorization, and without a valid
// notification body) gets the 401 it requires rather than a 400 (ADR-0007).
app.MapPost("/notifications", async (
HttpRequest request,
NotificationProjector projector,
CancellationToken ct) =>
{
if (request.Headers.Authorization != webhookToken)
return Results.Unauthorized();
var dto = await JsonSerializer.DeserializeAsync<NotificationDto>(
request.Body, SerializerOptions, ct);
if (dto is null)
return Results.BadRequest();
await projector.HandleAsync(dto.ToNotification(), ct);
return Results.NoContent();
});
// Admin: rebuild the projection from the durable notification log (PRD §8.4 — rebuildable).
app.MapPost("/admin/rebuild", async (NotificationProjector projector, CancellationToken ct) =>
{
await projector.RebuildAsync(ct);
return Results.NoContent();
});
await app.RunAsync();
/// <summary>The NRC notification body, as Open Notificaties POSTs it. Only the fields the
/// projection needs are bound; <c>aanmaakdatum</c>/<c>kenmerken</c> are ignored for the minimal slice.</summary>
public sealed record NotificationDto(string Kanaal, string Resource, string Actie, Uri ResourceUrl, Uri? HoofdObject = null)
{
public Notification ToNotification() => new(Kanaal, Resource, Actie, ResourceUrl, HoofdObject);
}
public partial class Program
{
// NRC sends camelCase JSON; match it case-insensitively.
private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.Web);
}