6 Commits

Include renames

Author	SHA1	Message	Date
not	5494363221	fix(k8s): keep Keycloak's backchannel URLs https behind the labs Caddy (refs #177) (#180) ... ## What & why Follow-up to #179. After login through `https://big-mijn.labs.respellion.tech`, the browser blocked the token request as mixed content. `KC_HOSTNAME_BACKCHANNEL_DYNAMIC=true` makes Keycloak build its token, userinfo and certs URLs from the incoming request. Behind the labs Caddy that request arrives as plain `http`, so the discovery document listed `http://big-auth…/token`. `KC_PROXY_HEADERS=xforwarded` makes Keycloak trust the `X-Forwarded-Proto: https` that Caddy sends. In-cluster calls (the BFF → `keycloak:8080`) carry no such header, so they are unchanged, and so is the localhost/NodePort setup. Refs #177 ## Definition of Done - [x] Linked Gitea issue (above). - [ ] Failing test committed before the implementation. *(One env var, verified live instead.)* - [x] Conventional Commits referencing the issue (`refs #NN`). - [ ] CI green - [x] Docs updated if behaviour, contracts, or operations changed. *(Comment in values.yaml.)* ## Notes for reviewers - I already applied this to the running cluster with `kubectl set env` and checked it. Both realms' discovery documents now have 0 `http://` URLs, and a `jan-burger` token from the public Keycloak still gets a 204 from the BFF. Merging keeps the next deploy from reverting it. - Keycloak now trusts `X-Forwarded-*` from anything that can reach it. Its only entry points are in-cluster callers and the NodePort, which the reverse tunnel exposes only to Caddy. `KC_PROXY_TRUSTED_ADDRESSES` could narrow that if the NodePort is ever exposed more widely. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #180	2026-09-25 12:30:15 +00:00
not	0074a1bff3	feat(k8s): optionally auto-fill the medewerker OTP step for the public demo (refs #177) (#181) ... ## What & why For the public demo on `big-behandel` / `big-beheer`, visitors should see MFA being enforced without needing an authenticator app. This adds an opt-in Keycloak theme that fills in and submits the medewerker OTP code itself. - **Theme as real files in `infra/keycloak/themes/big-demo/`**, next to the realms: - `login/theme.properties`: `keycloak.v2` plus `scripts=js/otp-autofill.js`. I checked the 26.1 source: `keycloak.v2` loads theme `scripts` and sets none of its own. - `login/resources/js/otp-autofill.js`: on the OTP page, computes the code (RFC 6238, Keycloak's default policy) from the fixture secret `BIGMEDEWERKEROTPSEED` and submits it. - `account`, `admin`, `email`: plain children of Keycloak 26's defaults. Without them the account console returns 500 (see notes). - **Seeded like every other file input:** `infra/helm/seed-configmaps.sh` creates the `rr-kc-theme` ConfigMap, and the chart mounts it as a directory. The podspec gains `items` so flat ConfigMap keys map to theme paths. Keycloak runs `start-dev` (no theme cache), so edits show up about a minute after a reseed. - **Switch:** `demo.otpAutofill` only decides whether `KC_SPI_THEME_DEFAULT=big-demo` is set. `big.env` now skips env values that render empty, and no existing env var is empty. **Off, the render is identical to main except for that one missing variable,** so Keycloak keeps its stock theme. The realm JSONs are untouched, so compose and the e2e tests still require a code. - **Single-use codes:** a second login in the same 30 s window spends the next counter, as `nextUnusedCounter` does in the e2e. Past that it only fills in the field and doesn't submit, so a rejected code can't loop. - **Deploy workflow:** repo variable `OTP_AUTOFILL=true` → `--set demo.otpAutofill=true`. Flipping it changes the pod's env, so Keycloak restarts. Refs #177 ## Definition of Done - [x] Linked Gitea issue (above). - [ ] Failing test committed before the implementation. *(Not done; checks below.)* - [x] Conventional Commits referencing the issue (`refs #NN`). - [ ] CI green - [x] `docker compose up` unaffected (chart only). - [x] Docs updated (Talos runbook, "Publishing through the labs Caddy"). - [ ] ADR. The fixture-secret trade-off is ADR-0031's; this only automates typing it in. ## Notes for reviewers - **Tested on the live cluster.** I patched the running Keycloak with the rendered theme (autofill on) and ran real headless Chromium logins against the public hosts: - `merel-behandelaar` on big-behandel: only username and password typed. The OTP page loaded the script, submitted by itself, and the user landed in the Werkbak. - `jan-burger` on big-mijn still logs in (regression check). - `/realms/medewerker/account/` returns 200. - **Account console 500, found live and fixed in the second commit.** `KC_SPI_THEME_DEFAULT` applies to every theme type, and Keycloak does *not* fall back for a type the theme lacks (`NullPointerException ... "theme" is null`). `big-demo` now declares login, account, admin and email, each a plain child of Keycloak 26's default. It's one ConfigMap mounted as a directory; the podspec gains `items` for that. - **Keycloak restarts cause about 5 minutes of BFF 401s.** This is not caused by this PR, but you'll see it whenever Keycloak restarts. Dev-mode Keycloak makes new signing keys on each boot, and the BFF refreshes its cached keys at most every 5 minutes. Seen live: 401 right after the restart, 204 about 4½ minutes later. Flipping `OTP_AUTOFILL` restarts Keycloak, so expect this briefly. - `make k8s-lint` and `make k8s-drift` pass. The rendered script's code matches `infra/keycloak/check_realms.py otp`. - **Security:** with it on, the public behandel and beheer portals are protected only by the committed password `test123`. That's intentional for synthetic demo data. Never enable it anywhere real. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #181	2026-09-25 11:34:30 +00:00
not	594fdde227	fix(infra): cap celery workers at 2 so the shared node stops OOM-killing CI (closes #182) (#183) ... ## What & why `verify-stack` is being killed by the OOM controller on the shared Talos node, on main (run 827) and on #180 (run 830). The cause is Celery: with `CELERY_WORKER_CONCURRENCY` unset, `oz-celery` and `nrc-celery` each fork one worker per CPU. That's 22 each on the lab node, 49 Celery processes at about 225 MB apiece. Details and the kernel log evidence are in #182. This sets `CELERY_WORKER_CONCURRENCY: "2"` in the oz and nrc env groups: - **compose** (`&oz-env`, `&nrc-env`): what `verify-stack` starts inside `dind`. - **chart** (`envGroups.oz` / `.nrc`): the deployed demo on the same node. Both images' `/celery_worker.sh` honour the variable; I checked in the running pods. Web, init and beat containers share the anchors and ignore it. `objecten-celery` already defaults to 1. Closes #182 ## Definition of Done - [x] Linked Gitea issue (above). - [ ] Failing test committed before the implementation. *(Resource setting; the evidence is the OOM log in #182.)* - [x] Conventional Commits referencing the issue (`refs #NN`). - [ ] CI green. This PR's `verify-stack` run is the check. - [x] `docker compose config` renders the variable into all 7 services on the two anchors. - [x] Docs: comments next to the setting, following the existing uWSGI notes. ## Notes for reviewers - `make k8s-lint` and `make k8s-drift` pass. - **Not applied live.** I couldn't patch the running cluster from my session. After merge, the deploy updates the `oz-env` / `nrc-env` ConfigMaps. The celery pods only pick that up on restart, and the deploy step restarts only this repo's nine services. So run once: `kubectl -n big rollout restart deploy/oz-celery deploy/nrc-celery` - **Why 2 and not 1:** this matches `UWSGI_THREADS: "2"`, and it keeps one notification delivery from blocking behind a slow task. It cuts roughly 40 processes, about 9 GB RSS (less in practice, because forked workers share pages). - **Longer term:** CI and the demo share one 15 GB VM. Resource requests on the runner, or moving the runner off the node, would stop one from starving the other. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #183	2026-09-25 11:11:45 +00:00
not	804031eeb8	feat(k8s): publish the portals through the labs Caddy (refs #177) (#179) ... ## What & why Makes the portals reachable on real hostnames through the Caddy that already fronts `*.labs.respellion.tech`, instead of five SSH port-forwards: | URL | Service | |---|---| | `https://big-register.labs.respellion.tech` | openbaar | | `https://big-mijn.labs.respellion.tech` | self-service | | `https://big-behandel.labs.respellion.tech` | behandel | | `https://big-beheer.labs.respellion.tech` | beheer | | `https://big-auth.labs.respellion.tech` | Keycloak (`/admin` blocked) | Chain: browser → labs Caddy (TLS) → `openssh-server` container → reverse SSH tunnel → Fedora host → Talos NodePorts. The Caddy routes and the tunnel unit are already on `main` in the Infra repo (`infra/development/`). This repo's part: - **Chart:** a `keycloakUrl` value. When set it replaces `host` + Keycloak's NodePort as the pinned issuer (`KC_HOSTNAME`) and the portals' OIDC authority. Both now come from one helper, `big.keycloakUrl`, so they can't drift apart (ADR-0010). Empty = rendered output identical to today. - **Deploy workflow:** passes the `KEYCLOAK_URL` repo variable as `--set keycloakUrl=…`. - **Runbook:** new section "Publishing through the labs Caddy". Refs #177 ## Definition of Done - [x] Linked Gitea issue (above). - [ ] Failing test committed before the implementation. *(Infra/config change, no test added.)* - [x] Implementation makes the test pass; refactor commit if structure improved. - [x] Conventional Commits referencing the issue (`refs #NN`). - [ ] CI green — all Gitea Actions jobs (or `make ci` green while no runner exists). - [x] `docker compose up` from a fresh clone reaches green health checks within 3 minutes. *(Compose untouched.)* - [x] Docs updated if behaviour, contracts, or operations changed. - [ ] ADR added in `docs/architecture/` if a non-obvious decision was made. - [ ] Demo note in `docs/demo-script.md` if user-visible. ## Notes for reviewers - **This takes the option #177 rejects.** #177 proposes an in-cluster Caddy edge (branch `feat/177-public-tls-edge`). This PR uses the existing labs Caddy instead, because it already holds 80/443 and the wildcard certificate. So it only *refs* #177. If we go this way, #177's ADR should record the host-Caddy option instead. - `make k8s-lint` and `infra/check-docs-nav.py` pass. I rendered the chart with and without `keycloakUrl`: empty gives the same output as before; set, it gives `https://big-auth.labs.respellion.tech` for both the issuer and the authority. - Once `KEYCLOAK_URL` is set, the `localhost` port-forward workflow (runbook §5) no longer logs in, because the issuer is a single string. - The portals are public, with no Azure `authorize` in front of them the way `marketing` has one. The test users use `test123`. - Rollout after merge: install `big-portals-tunnel.service` on the Fedora host, run `docker compose up -d caddy` on the labs server, then set the `KEYCLOAK_URL` variable. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #179	2026-09-25 08:16:37 +00:00
not	9d7e8e5b65	ci(k8s): gate the Helm chart in CI + a compose↔chart drift check (closes #168) (#171) ... ## What & why The Helm chart landed in #167 with two gaps written into ADR-0033: `make k8s-lint` existed but no CI job ran it, and *"a second deployment description to keep in step with compose — nothing enforces that today; a drift check belongs in CI (follow-up)"*. Both are closed here. **`make k8s-drift`** (`infra/helm/check-drift.py`, stdlib only) compares what each stack actually deploys rather than diffing two files that differ by design: workload names and resolved container images, taken from `docker compose config --format json` and a rendered chart. The six differences that exist today are declared in `DEVIATIONS` with the reason each was forced — the four `*-init` Django services folded into their web pods, and the two bootstrap Jobs compose runs from the host — so only a *new* difference fails. **A `k8s` CI job** runs `k8s-lint` then `k8s-drift` on every push and PR. No cluster, no marketplace action: helm is fetched as the pinned static binary the Talos runbook already gives developers. Closes #168 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation — the red commit reports all six real differences; the green commit declares them. - [x] Implementation makes the test pass. - [x] Conventional Commits referencing the issue (`refs #168`). - [x] CI green — awaiting the run on this PR (`make k8s-lint` and `make k8s-drift` pass locally). - [x] `docker compose up` unaffected — no service, image or compose file is touched. - [x] Docs updated — `docs/runbooks/ci.md` (job table + the one place local and CI now differ), `docs/runbooks/kubernetes-talos.md` §7/§"not ported", and ADR-0033's cost note. - [x] No ADR needed: no new dependency (python stdlib, and helm/docker were already prerequisites of the `k8s-*` targets), no boundary moved, no §8 rule bent. - [x] Not user-visible, so no demo note. ## Notes for reviewers Verified by hand that both drift classes fail the check, not just that it passes today: - bumping `OPENZAAK_TAG` in compose alone → reports `openzaak` and `oz-celery` with both image strings; - adding a workload to `values.yaml` alone → reports it by name. Deliberate limits (there is a `ponytail:` note in the script): - **Names and images only**, as sets — no per-workload env, ports or volumes. Those differ by design in four documented places, so comparing them would mean re-encoding every deviation field by field for very little more signal. - **The three observability workloads are rendered with `enabled=true`** by the check, even though both stacks default them off, so their images can't drift unwatched. - **`k8s-lint`/`k8s-drift` are not in `make ci`**, to avoid making `helm` a hard prerequisite for everyone. That is now the only local/CI difference; it's called out in `docs/runbooks/ci.md`. Follow-ups filed while reviewing the chart, not addressed here: #169 (the published docs omit every ADR after 0010 and all runbooks but `ci.md`) and #170 (the production-posture ADR #25 asked for — secrets are still plain text in `values.yaml`).Reviewed-on: #171	2026-09-18 13:25:24 +00:00
not	1dd8bd4e1b	S-24/#25 · Helm chart + Kubernetes deployment, and Caddy for the portals (#166) (#167) ... ## What & why Two changes, made and verified together on a real cluster. **S-24 / #25 — a Helm chart for the platform.** One chart, `infra/helm/big-reference`, whose `values.yaml` is a near-literal transcription of `infra/docker-compose.yml`, rendered by three generic templates (Deployment, Job, Service) over a `workloads` map. Adding a service is a values edit. `make k8s-lint` renders and schema-checks the whole stack without a cluster. The issue asked for a *sketch*; this is deployed and verified end to end (see below), which is more than it asked for — the part it asked for that is **not** here is the production-posture write-up (HA, secrets, backup), see Known gaps. **#166 — Caddy replaces nginx in the portals.** nginx resolves a variable `proxy_pass` upstream itself, using only the `resolver` directive and never `/etc/resolv.conf`'s search domains. That had cost two workarounds in one script: rewriting the resolver address for rootless podman, and injecting a full FQDN so the bare `bff` name could resolve on Kubernetes. Caddy dials per request through the system resolver, so `reverse_proxy bff:8080` works on every engine unchanged; `apps/portal-nginx-resolver.sh` and the chart's `BFF_HOST` env are deleted. Closes #25 Closes #166 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation — twice: the Caddyfile contract test before the Caddyfiles, `make k8s-lint` before the chart. - [x] Implementation makes the test pass. - [x] Conventional Commits referencing the issues (`refs #25` / `refs #166`). - [ ] CI green — awaiting the run on this PR (`make k8s-lint`, `dotnet format` and the new unit self-check pass locally; the compose e2e and mutation lanes are CI's). - [ ] `docker compose up` from a fresh clone reaches green health checks within 3 minutes — the portal images were rebuilt and verified standalone, but a full `make up` run has not been done on this branch. Please confirm in review or let CI's smoke test speak. - [x] Docs updated — `docs/runbooks/kubernetes-talos.md` (new), `frontend-decisions.md`, `demo-script.md`, and the docs that named nginx. - [x] ADR added — ADR-0033 (chart) and ADR-0034 (Caddy). - [ ] Demo note in `docs/demo-script.md` — not added: the deployment target is not a user-visible slice, and the Caddy swap is invisible to the demo script beyond the wording fix included here. ## How it was verified Brought up from scratch on a single-node Talos v1.14.0 VM (6 vCPU / 10 GB, virtio disk) under virt-manager: **29 pods ready and four bootstrap Jobs complete in under three minutes, zero restarts**, using ~4.4 GB of the VM's 10 GB. - Full Common Ground path: portal Caddy → BFF → domain → Flowable → ACL → OpenZaak + Objecten → NRC → event-subscriber → projection → public register (`INGEDIEND`, reference matching the submitted registration). - Werkbak read with an MFA'd medewerker token → 200. - The browser flow driven with Playwright against `http://localhost:30140`: secure context, `crypto.subtle` present, Keycloak form reached, login completed, **no console errors**. - Routing checked against a stub BFF: SPA fallback serves deep links, each portal proxies its own groups, and a portal does *not* proxy a neighbour's group. ## Notes for reviewers Three bugs this shook out, each fixed at the cause rather than the symptom: 1. **`command` vs `args`.** Compose's `command:` replaces the image CMD; Kubernetes' replaces the ENTRYPOINT. Transcribing one to the other broke every upstream image that relies on its entrypoint — postgres refused to run as root, Keycloak tried to exec `start-dev`. The chart now `fail`s at render time on `command`. 2. **Concurrent migrations.** Both `/setup_configuration.sh` and `/start.sh` run `manage.py migrate`; compose serialises them with `depends_on`, Kubernetes has no such edge, so the init Job and its web pod raced (`relation "zgw_consumers_service" already exists`). The four Django services now do both steps in order in the web pod — which also deletes four workloads. 3. **`emptyDir` databases are wiped by any pod-template change.** `make k8s-reseed` now also restarts `event-subscriber` and `projection-api`, which create the projection schema on start and otherwise keep writing to a schema-less database. Known gaps / follow-ups: - **Secrets.** `values.yaml` carries the dev credentials in plain text (`admin/admin`, the ZGW client secret, the two Objecten tokens) and the chart has no `Secret` objects. Fine for a laptop demo, and exactly what #25's "production posture" ADR should address — I suggest a follow-up issue rather than stretching this PR. - **No CI gate for the chart yet.** `make k8s-lint` exists but is not wired into `.gitea/workflows/ci.yaml`, and nothing enforces that the chart and the compose file stay in step. Worth a small follow-up. - **This is two slices in one PR.** They were built and verified together and the diff is entangled (the chart was written against Caddy from the start), so splitting now would mean re-creating an nginx-shaped chart to throw away. Happy to split if you'd rather. - **Rebased onto #161** (merged as #165) rather than merged, to keep the history linear. One conflict, in the `unit:` target where both branches add a self-check line — resolved by keeping both. #161's `infra/host-browser.yml` arrived with `/usr/share/nginx/html/config.json` and is fixed to `/usr/share/caddy/` inside the `feat(portals)` commit, so no commit on this branch leaves that overlay pointing at a path the images no longer have.Reviewed-on: #167	2026-09-10 08:53:58 +00:00