diff --git a/docs/architecture/adr-0023-observability-stack.md b/docs/architecture/adr-0023-observability-stack.md index 7adb034..da7f6cf 100644 --- a/docs/architecture/adr-0023-observability-stack.md +++ b/docs/architecture/adr-0023-observability-stack.md @@ -67,6 +67,14 @@ itself, so no in-image healthcheck tool is required. - Three more images built each CI run (kept small; not on the health-gate list). - Storage is ephemeral container fs — a demo backplane, not a retention target. Object storage for Tempo / remote-write for Prometheus is a later concern. +- Tempo runs **single-binary**, so its distributor and ingester are one process and + some of its distributed-mode machinery is not just redundant but harmful. Its + ingester-pool health check is disabled (`ingester_client.pool_config`) because with + a single in-process ingester the check can never route around a failure — a 1s + loopback-gRPC deadline missed under CI load only evicted the one ingester and made + Tempo drop spans, which is how `verify-tracing` flaked (#156). Expect the same + shape from other distributed-mode knobs if we tune them; the fix is to switch to + real multi-ingester Tempo, not to re-enable them here. ## Coupling rules touched (CLAUDE.md §8)