# ADR-0035: The deployed stack is published through the existing labs Caddy

- **Status:** Accepted
- **Date:** 2026-09-25
- **Deciders:** Respellion engineering
- **Slice:** [#177](https://git.labs.respellion.tech/eho/register-referentie/issues/177) —
  that issue proposed the opposite (an in-cluster Caddy edge); this ADR records why the
  host-side option won. Implemented in #179, #180 and #181.

## Context

The stack deploys to a single-node Talos VM (ADR-0033, #175). Until now it was only usable
through five SSH port-forwards: the portals' OIDC flow uses PKCE, PKCE needs
`crypto.subtle`, and browsers expose that only in a **secure context**, meaning HTTPS or a
`localhost` origin. A NodePort on the VM's address is neither. We want a URL a demo
audience can simply open.

Three facts about where things run shape the answer:

- The Talos VM is a libvirt guest on a **Fedora hypervisor in the office**, behind NAT
  with no public address. The only way in from outside is an existing reverse SSH tunnel
  (`autossh-reverse-tunnel.service`) into an `openssh-server` container on the labs
  server.
- The **labs server** (public IP) already runs Caddy for `*.labs.respellion.tech`, with
  the wildcard certificate (DNS-01 via Cloudflare) and ports 80/443. Every other labs
  service is published there (repo `Infra`, `infra/development/`).
- #177 proposed a Caddy **inside the cluster**, fed by a layer-4 forward on the host, so
  that routing and certificates would be cluster state. That assumes the public IP is on
  the hypervisor. It isn't: the hypervisor has no inbound path, and 80/443 on the labs
  server are already taken by the labs Caddy.

## Decision

**Publish the portals and Keycloak through the existing labs Caddy. Carry the traffic to
the cluster over a second reverse SSH tunnel from the hypervisor.**

```
browser ─https─▶ labs Caddy ─▶ openssh-server:3014x/30180
        ─reverse SSH tunnel─▶ Fedora hypervisor ─▶ Talos NodePorts
```

- **Hostnames** under the existing wildcard: `big-register` (openbaar), `big-mijn`
  (self-service), `big-behandel`, `big-beheer`, and `big-auth` (Keycloak, with `/admin*`
  answered 404).
- **Tunnel:** `big-portals-tunnel.service` on the hypervisor (repo `Infra`)
  reverse-forwards the five browser-facing NodePorts into `openssh-server`. It is
  separate from the access tunnel on `:6667`, so a failed forward can't cut SSH access.
  Caddy joins the `openssh_default` network to reach the tunnel ends.
- **Keycloak's issuer** is the public origin. The chart value `keycloakUrl` replaces
  `host` + NodePort in one helper, `big.keycloakUrl`, which feeds both `KC_HOSTNAME` and
  the portals' `config.json` authority, so the two cannot drift (ADR-0010). The deploy
  workflow sets it from the `KEYCLOAK_URL` repository variable.
- **`KC_PROXY_HEADERS=xforwarded`:** `KC_HOSTNAME_BACKCHANNEL_DYNAMIC` builds the token,
  userinfo and certs URLs from the request. That request reaches Keycloak as plain HTTP,
  so the URLs came out `http://` and browsers blocked them as mixed content. Trusting
  Caddy's `X-Forwarded-Proto` keeps them HTTPS. In-cluster calls send no such header and
  still use `keycloak:8080`.
- **Demo MFA (optional):** `demo.otpAutofill` (`OTP_AUTOFILL`) makes the `big-demo` theme
  (`infra/keycloak/themes/big-demo`) Keycloak's default. Its script fills in and submits
  the medewerker OTP from the fixture secret (ADR-0031), so the step is visibly enforced
  without an authenticator. It is off by default.

### Alternatives considered

- **In-cluster Caddy edge (#177, PR #178).** It would keep routes and certificates in
  cluster state. But it needs a public inbound path to the hypervisor that doesn't exist,
  plus a second certificate authority beside the labs Caddy, which already holds the
  wildcard. Closed unmerged.
- **Port-forward on the office router to the hypervisor.** This opens the office network
  itself to the internet. Rejected.
- **Move the cluster to a host with a public IP.** It would remove the tunnel, but it's a
  bigger change than publishing one demo. It remains the natural step if the stack
  outgrows a lab VM.
- **Keep the SSH port-forwards.** Fine for one developer, but not something you can send
  to someone.

## Consequences

**Positive**

- Real hostnames and HTTPS, so PKCE works in any browser with no client-side setup.
- No new certificate handling: the labs Caddy's wildcard covers the new hosts.
- The chart stays edge-agnostic. With `keycloakUrl` empty it renders exactly as before,
  so compose, CI and the `localhost` workflow are untouched.

**Negative / costs**

- **Routing lives outside the cluster**, in the Infra repo's Caddyfile. That is exactly
  what #177 wanted to avoid. Adding a portal means changing three places: a NodePort in
  the chart, a forward in the tunnel unit, and a host in the Caddyfile.
- **Two SSH hops in the data path.** If the hypervisor or the tunnel is down, the
  portals return 502 even though the cluster is healthy.
- **One issuer string.** With `keycloakUrl` set, the `localhost` port-forward workflow
  (runbook §5) can no longer log in.
- **Keycloak trusts `X-Forwarded-*`** from anything that reaches it. Today that is only
  in-cluster callers and the tunnel. `KC_PROXY_TRUSTED_ADDRESSES` can narrow it if the
  NodePort is ever exposed more widely.
- **The portals are public.** Anyone with the link can log in with the committed test
  credentials, and with `OTP_AUTOFILL` on, no second factor stands in the way. That is
  acceptable for synthetic data. Put the labs Caddy's Azure `authorize` in front of the
  `big-*` hosts if the audience must be restricted.

**Follow-up**

- Runbook: `docs/runbooks/kubernetes-talos.md`, "Publishing through the labs Caddy".
- Dev-mode Keycloak generates new signing keys on every restart, and the BFF re-fetches
  them at most every 5 minutes, so expect a few minutes of 401s after a Keycloak restart.
  Persisting Keycloak's database (runbook §6) would remove that.