docs(arch): ADR-0035 — publish the stack through the existing labs Caddy (closes #177)

Records the decision #177 asked for, the other way round: the hypervisor has no
inbound path and the labs Caddy already holds 80/443 and the wildcard cert, so
the portals go through it over a reverse SSH tunnel instead of an in-cluster
edge. Covers keycloakUrl, KC_PROXY_HEADERS and the optional demo OTP autofill.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.gitea/workflows/ci.yaml

@@ -207,8 +207,14 @@ jobs:
   # dispatched (gitea-actions-gotchas.md §7). Default `if: success()` dispatches normally. Cost: a
   # failing mutation ratchet now skips verify-stack instead of running it anyway; the fix-and-re-push
   # re-run exercises verify-stack, so we still get the signal.
+  #
+  # Main only, not on PRs: the runner shares the lab node with the deployed stack, and a second
+  # full stack per PR was what got the runner OOM-killed (#182). PRs still gate on every job above;
+  # the live-stack check runs once per merge. A plain event `if` keeps the implicit success(), so it
+  # is not the status-function case from gotchas §7.
   verify-stack:
     needs: [mutation]
+    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
     runs-on: ubuntu-latest
     steps:
       - uses: https://github.com/actions/checkout@v4

.gitea/workflows/deploy.yaml

@@ -31,6 +31,9 @@ jobs:
       # origin, e.g. https://big-auth.labs.respellion.tech (runbook, "Publishing
       # through the labs Caddy").
       KEYCLOAK_URL: ${{ vars.KEYCLOAK_URL }}
+      # `true` fills in the medewerker OTP step for the public demo (chart value
+      # demo.otpAutofill). The fixture secret is committed: demo only.
+      OTP_AUTOFILL: ${{ vars.OTP_AUTOFILL }}
     steps:
       - uses: https://github.com/actions/checkout@v4
 
@@ -97,7 +100,7 @@ jobs:
           make k8s-reseed \
             TALOS_HOST=${TALOS_HOST:-localhost} \
             K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500 \
-            K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL}"
+            K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}"
 
       # `dev` is a mutable tag and helm sees an unchanged pod template, so the
       # new images only land on a restart (pullPolicy is already Always).

docs/architecture/adr-0035-public-access-through-the-labs-caddy.md

@@ -0,0 +1,108 @@
+# ADR-0035: The deployed stack is published through the existing labs Caddy
+
+- **Status:** Accepted
+- **Date:** 2026-09-25
+- **Deciders:** Respellion engineering
+- **Slice:** [#177](https://git.labs.respellion.tech/eho/register-referentie/issues/177) —
+  that issue proposed the opposite (an in-cluster Caddy edge); this ADR records why the
+  host-side option won. Implemented in #179, #180 and #181.
+
+## Context
+
+The stack deploys to a single-node Talos VM (ADR-0033, #175). Until now it was only usable
+through five SSH port-forwards: the portals' OIDC flow uses PKCE, PKCE needs
+`crypto.subtle`, and browsers expose that only in a **secure context**, meaning HTTPS or a
+`localhost` origin. A NodePort on the VM's address is neither. We want a URL a demo
+audience can simply open.
+
+Three facts about where things run shape the answer:
+
+- The Talos VM is a libvirt guest on a **Fedora hypervisor in the office**, behind NAT
+  with no public address. The only way in from outside is an existing reverse SSH tunnel
+  (`autossh-reverse-tunnel.service`) into an `openssh-server` container on the labs
+  server.
+- The **labs server** (public IP) already runs Caddy for `*.labs.respellion.tech`, with
+  the wildcard certificate (DNS-01 via Cloudflare) and ports 80/443. Every other labs
+  service is published there (repo `Infra`, `infra/development/`).
+- #177 proposed a Caddy **inside the cluster**, fed by a layer-4 forward on the host, so
+  that routing and certificates would be cluster state. That assumes the public IP is on
+  the hypervisor. It isn't: the hypervisor has no inbound path, and 80/443 on the labs
+  server are already taken by the labs Caddy.
+
+## Decision
+
+**Publish the portals and Keycloak through the existing labs Caddy. Carry the traffic to
+the cluster over a second reverse SSH tunnel from the hypervisor.**
+
+```
+browser ─https─▶ labs Caddy ─▶ openssh-server:3014x/30180
+        ─reverse SSH tunnel─▶ Fedora hypervisor ─▶ Talos NodePorts
+```
+
+- **Hostnames** under the existing wildcard: `big-register` (openbaar), `big-mijn`
+  (self-service), `big-behandel`, `big-beheer`, and `big-auth` (Keycloak, with `/admin*`
+  answered 404).
+- **Tunnel:** `big-portals-tunnel.service` on the hypervisor (repo `Infra`)
+  reverse-forwards the five browser-facing NodePorts into `openssh-server`. It is
+  separate from the access tunnel on `:6667`, so a failed forward can't cut SSH access.
+  Caddy joins the `openssh_default` network to reach the tunnel ends.
+- **Keycloak's issuer** is the public origin. The chart value `keycloakUrl` replaces
+  `host` + NodePort in one helper, `big.keycloakUrl`, which feeds both `KC_HOSTNAME` and
+  the portals' `config.json` authority, so the two cannot drift (ADR-0010). The deploy
+  workflow sets it from the `KEYCLOAK_URL` repository variable.
+- **`KC_PROXY_HEADERS=xforwarded`:** `KC_HOSTNAME_BACKCHANNEL_DYNAMIC` builds the token,
+  userinfo and certs URLs from the request. That request reaches Keycloak as plain HTTP,
+  so the URLs came out `http://` and browsers blocked them as mixed content. Trusting
+  Caddy's `X-Forwarded-Proto` keeps them HTTPS. In-cluster calls send no such header and
+  still use `keycloak:8080`.
+- **Demo MFA (optional):** `demo.otpAutofill` (`OTP_AUTOFILL`) makes the `big-demo` theme
+  (`infra/keycloak/themes/big-demo`) Keycloak's default. Its script fills in and submits
+  the medewerker OTP from the fixture secret (ADR-0031), so the step is visibly enforced
+  without an authenticator. It is off by default.
+
+### Alternatives considered
+
+- **In-cluster Caddy edge (#177, PR #178).** It would keep routes and certificates in
+  cluster state. But it needs a public inbound path to the hypervisor that doesn't exist,
+  plus a second certificate authority beside the labs Caddy, which already holds the
+  wildcard. Closed unmerged.
+- **Port-forward on the office router to the hypervisor.** This opens the office network
+  itself to the internet. Rejected.
+- **Move the cluster to a host with a public IP.** It would remove the tunnel, but it's a
+  bigger change than publishing one demo. It remains the natural step if the stack
+  outgrows a lab VM.
+- **Keep the SSH port-forwards.** Fine for one developer, but not something you can send
+  to someone.
+
+## Consequences
+
+**Positive**
+
+- Real hostnames and HTTPS, so PKCE works in any browser with no client-side setup.
+- No new certificate handling: the labs Caddy's wildcard covers the new hosts.
+- The chart stays edge-agnostic. With `keycloakUrl` empty it renders exactly as before,
+  so compose, CI and the `localhost` workflow are untouched.
+
+**Negative / costs**
+
+- **Routing lives outside the cluster**, in the Infra repo's Caddyfile. That is exactly
+  what #177 wanted to avoid. Adding a portal means changing three places: a NodePort in
+  the chart, a forward in the tunnel unit, and a host in the Caddyfile.
+- **Two SSH hops in the data path.** If the hypervisor or the tunnel is down, the
+  portals return 502 even though the cluster is healthy.
+- **One issuer string.** With `keycloakUrl` set, the `localhost` port-forward workflow
+  (runbook §5) can no longer log in.
+- **Keycloak trusts `X-Forwarded-*`** from anything that reaches it. Today that is only
+  in-cluster callers and the tunnel. `KC_PROXY_TRUSTED_ADDRESSES` can narrow it if the
+  NodePort is ever exposed more widely.
+- **The portals are public.** Anyone with the link can log in with the committed test
+  credentials, and with `OTP_AUTOFILL` on, no second factor stands in the way. That is
+  acceptable for synthetic data. Put the labs Caddy's Azure `authorize` in front of the
+  `big-*` hosts if the audience must be restricted.
+
+**Follow-up**
+
+- Runbook: `docs/runbooks/kubernetes-talos.md`, "Publishing through the labs Caddy".
+- Dev-mode Keycloak generates new signing keys on every restart, and the BFF re-fetches
+  them at most every 5 minutes, so expect a few minutes of 401s after a Keycloak restart.
+  Persisting Keycloak's database (runbook §6) would remove that.

docs/runbooks/ci.md

@@ -21,7 +21,7 @@ and CI cannot drift:
 | `frontend` | `make frontend` → Nx lint/test/build for the four portals | pnpm + Node |
 | `k8s` | `make k8s-lint` (render + schema-check the Helm chart) → `make k8s-drift` (chart still describes the same stack as `infra/docker-compose.yml`) | pinned `helm` binary + `docker compose` |
 | `mutation` | `make mutation` → `dotnet tool restore` → `dotnet stryker` (ACL); uploads the HTML report as an artifact | .NET 10 SDK |
-| `verify-stack` | the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
+| `verify-stack` | **push to `main` only, skipped on PRs** (#182) — the single live-stack stage — steps: `make verify-up` (full stack up + health, the DoD smoke) → `make verify-acl` (ACL ↔ OpenZaak) → `make verify-nrc` (OpenZaak → NRC delivery) → `make down` | container engine + egress (base images, nuget, `selectielijst.openzaak.nl`) |
 
 > **Why one `verify-stack` job, not three.** The single self-hosted runner runs jobs
 > **sequentially**, so booting OpenZaak once (instead of once per check) is the

docs/runbooks/kubernetes-talos.md

@@ -403,6 +403,8 @@ upgrade path.
 
 ## Publishing through the labs Caddy
 
+Why this route and not an in-cluster edge: [ADR-0035](../architecture/adr-0035-public-access-through-the-labs-caddy.md).
+
 The portals can be reached on real hostnames through the Caddy that already fronts
 `*.labs.respellion.tech` (repo `Infra`, `infra/development/`). The chain:
 
@@ -430,6 +432,18 @@ make k8s-up TALOS_HOST=localhost K8S_REGISTRY=<TALOS_HOST>:30500 \
 For deploy-on-merge, set the repository variable `KEYCLOAK_URL` to the same value.
 With it set, the `localhost` port-forwards (§5) no longer log in: the issuer is one string.
 
+Staff logins still hit the enforced OTP step. For a demo, set the repository variable
+`OTP_AUTOFILL=true` (chart value `demo.otpAutofill`): Keycloak then uses the `big-demo`
+theme, which fills in and submits the code from the fixture secret, so the step is visible
+but needs no authenticator. Keycloak restarts when the value flips. Demo only — the secret
+is committed.
+
+The theme lives in `infra/keycloak/themes/big-demo/` and is seeded as the `rr-kc-theme`
+ConfigMap by `infra/helm/seed-configmaps.sh` on every deploy. Keycloak runs `start-dev`,
+which doesn't cache themes, so an edit shows up about a minute after the ConfigMap changes.
+A *new* theme file also needs a key in the seed script and a path in the keycloak `files`
+in `values.yaml`.
+
 One-time setup:
 
 1. Fedora host: install `infra/development/big-portals-tunnel.service` from the Infra repo

infra/helm/big-reference/templates/_helpers.tpl

@@ -94,6 +94,10 @@ volumes:
       {{- with .defaultMode }}
       defaultMode: {{ . }}
       {{- end }}
+      {{- with .items }}
+      items:
+{{- toYaml . | nindent 8 }}
+      {{- end }}
 {{- end }}
 {{- with $w.data }}
   - name: data
@@ -125,13 +129,17 @@ volumes:
 {{/*
 Env list from a map. Every value is run through `tpl`, so values.yaml can name
 cluster-internal hosts ({{ .Release.Namespace }}) and the node address
-({{ .Values.host }}) without the chart hard-coding either.
+({{ .Values.host }}) without the chart hard-coding either. A value that renders
+empty is left out, which is how a setting is made conditional on a chart value.
 */}}
 {{- define "big.env" -}}
 {{- $root := index . 0 -}}
 {{- range $k, $v := index . 1 }}
+{{- $val := tpl (toString $v) $root }}
+{{- if $val }}
 - name: {{ $k }}
-  value: {{ tpl (toString $v) $root | quote }}
+  value: {{ $val | quote }}
+{{- end }}
 {{- end }}
 {{- end -}}
 

infra/helm/big-reference/values.yaml

@@ -30,6 +30,12 @@ host: 192.168.122.100
 # portals' authority (runbook, "Publishing through the labs Caddy").
 keycloakUrl: ""
 
+demo:
+  # Fill in and submit the medewerker OTP step from the fixture secret, so a public
+  # demo shows MFA enforced without an authenticator: makes the big-demo theme
+  # (infra/keycloak/themes/big-demo) Keycloak's default. Demo only: the secret is committed.
+  otpAutofill: false
+
 # Set when pulling from a private registry (e.g. the Gitea Container Registry).
 imagePullSecrets: []
 
@@ -277,11 +283,29 @@ workloads:
       # this issuer back, which is what browser tokens carry (infra/host-browser.yml).
       KC_HOSTNAME: '{{ include "big.keycloakUrl" . }}'
       KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
+      # Only rendered with demo.otpAutofill (big.env skips empty values); off, Keycloak
+      # keeps its stock theme and the mounted big-demo theme is unused.
+      KC_SPI_THEME_DEFAULT: '{{ if .Values.demo.otpAutofill }}big-demo{{ end }}'
+      # Behind a TLS proxy (keycloakUrl) the dynamic backchannel URLs — token,
+      # userinfo, certs — take their scheme from the request, which reaches Keycloak
+      # as plain http; trusting X-Forwarded-Proto keeps them https so the browser
+      # doesn't block them as mixed content. In-cluster calls send no such header.
+      KC_PROXY_HEADERS: xforwarded
     ports: [{ name: http, port: 8080 }]
     # TCP, not /health/ready on the management port: nothing here gates on realm
     # import, and a wrong health path would leave the Service with no endpoints.
     probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 15 }
-    files: [{ configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }]
+    files:
+      - { configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }
+      # infra/keycloak/themes/big-demo, seeded by infra/helm/seed-configmaps.sh.
+      - configMap: rr-kc-theme
+        mountPath: /opt/keycloak/themes/big-demo
+        items:
+          - { key: login.properties, path: login/theme.properties }
+          - { key: otp-autofill.js, path: login/resources/js/otp-autofill.js }
+          - { key: account.properties, path: account/theme.properties }
+          - { key: admin.properties, path: admin/theme.properties }
+          - { key: email.properties, path: email/theme.properties }
 
   # ── Flowable (S-03) ─────────────────────────────────────────────────────────
   flowable-db:

infra/helm/seed-configmaps.sh

@@ -30,6 +30,15 @@ seed() {  # name  <kubectl --from-file args...>
 seed rr-oz-config            --from-file="$repo/infra/openzaak/setup_configuration/"
 seed rr-nrc-config           --from-file="$repo/infra/opennotificaties/setup_configuration/"
 seed rr-kc-realms            --from-file="$repo/infra/keycloak/realms/"
+# The big-demo login theme (demo.otpAutofill). ConfigMap keys are flat, so each
+# file gets a key here and its path back in the keycloak `files` in values.yaml.
+theme="$repo/infra/keycloak/themes/big-demo"
+seed rr-kc-theme \
+  --from-file=login.properties="$theme/login/theme.properties" \
+  --from-file=otp-autofill.js="$theme/login/resources/js/otp-autofill.js" \
+  --from-file=account.properties="$theme/account/theme.properties" \
+  --from-file=admin.properties="$theme/admin/theme.properties" \
+  --from-file=email.properties="$theme/email/theme.properties"
 seed rr-objecttypen-config   --from-file="$repo/infra/objecttypen/setup_configuration/"
 seed rr-objecten-config      --from-file="$repo/infra/objecten/setup_configuration/"
 # register.py + the RegisterRecord JSON schema (the __pycache__ dir is skipped:

infra/keycloak/themes/big-demo/account/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the account page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak.v3

infra/keycloak/themes/big-demo/admin/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the admin page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak.v2

infra/keycloak/themes/big-demo/email/theme.properties

@@ -0,0 +1,4 @@
+# The chart makes big-demo the default for every theme type, and Keycloak does not
+# fall back for a type a theme lacks (the email page then fails), so each type is
+# declared as a plain child of Keycloak 26's own default.
+parent=keycloak

infra/keycloak/themes/big-demo/login/resources/js/otp-autofill.js

@@ -0,0 +1,24 @@
+// RFC 6238 with Keycloak's default policy (HmacSHA1, 6 digits, 30 s) over the
+// raw bytes of the medewerker fixture secret — same as tests/e2e/keycloak-login.ts.
+document.addEventListener('DOMContentLoaded', async () => {
+  const input = document.querySelector('input[name="otp"]');
+  if (!input || !input.form) return;
+  const key = await crypto.subtle.importKey('raw',
+    new TextEncoder().encode('BIGMEDEWERKEROTPSEED'), { name: 'HMAC', hash: 'SHA-1' }, false, ['sign']);
+  // A code is single-use, so a second login in the same window spends the next
+  // counter (Keycloak's look-ahead accepts it). Past that, fill but don't submit,
+  // so a rejected code can't turn into a submit loop.
+  const now = Math.floor(Date.now() / 30000);
+  let last = -1;
+  try { last = Number(sessionStorage.getItem('big-otp-counter')) || -1; } catch {}
+  const counter = Math.max(now, last + 1);
+  const msg = new DataView(new ArrayBuffer(8));
+  msg.setBigUint64(0, BigInt(counter));
+  const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, msg.buffer));
+  const o = mac[19] & 0x0f;
+  const n = ((mac[o] & 0x7f) << 24 | mac[o + 1] << 16 | mac[o + 2] << 8 | mac[o + 3]) % 1e6;
+  input.value = String(n).padStart(6, '0');
+  if (counter > now + 1) return;
+  try { sessionStorage.setItem('big-otp-counter', String(counter)); } catch {}
+  input.form.requestSubmit();
+});

infra/keycloak/themes/big-demo/login/theme.properties

@@ -0,0 +1,11 @@
+# Demo login theme for the public Talos deployment: keycloak.v2 plus a script that
+# fills in and submits the medewerker OTP step from the committed fixture secret
+# (docs/runbooks/keycloak.md). Only used when the chart's demo.otpAutofill is on —
+# it then becomes Keycloak's default theme. Never enable it anywhere real.
+#
+# Add styles, messages or template overrides here as in any Keycloak theme
+# (https://www.keycloak.org/ui-customization/themes); new files must also be
+# listed in infra/helm/seed-configmaps.sh and the keycloak `files` in values.yaml.
+parent=keycloak.v2
+import=common/keycloak
+scripts=js/otp-autofill.js

mkdocs.yml

@@ -56,6 +56,7 @@ nav:
       - "ADR-0032: Werkbak live refresh": architecture/adr-0032-werkbak-live-refresh.md
       - "ADR-0033: Kubernetes via one Helm chart": architecture/adr-0033-kubernetes-via-one-helm-chart.md
       - "ADR-0034: Caddy serves the portals": architecture/adr-0034-caddy-serves-the-portals.md
+      - "ADR-0035: Public access through the labs Caddy": architecture/adr-0035-public-access-through-the-labs-caddy.md
   - FDS-architectuur:
       - Overzicht: architecture/fds/README.md
       - Componentview (L3): architecture/fds/c4-component-view.md