## What & why Two changes, made and verified together on a real cluster. **S-24 / #25 — a Helm chart for the platform.** One chart, `infra/helm/big-reference`, whose `values.yaml` is a near-literal transcription of `infra/docker-compose.yml`, rendered by three generic templates (Deployment, Job, Service) over a `workloads` map. Adding a service is a values edit. `make k8s-lint` renders and schema-checks the whole stack without a cluster. The issue asked for a *sketch*; this is deployed and verified end to end (see below), which is more than it asked for — the part it asked for that is **not** here is the production-posture write-up (HA, secrets, backup), see Known gaps. **#166 — Caddy replaces nginx in the portals.** nginx resolves a variable `proxy_pass` upstream itself, using only the `resolver` directive and never `/etc/resolv.conf`'s search domains. That had cost two workarounds in one script: rewriting the resolver address for rootless podman, and injecting a full FQDN so the bare `bff` name could resolve on Kubernetes. Caddy dials per request through the system resolver, so `reverse_proxy bff:8080` works on every engine unchanged; `apps/portal-nginx-resolver.sh` and the chart's `BFF_HOST` env are deleted. Closes #25 Closes #166 ## Definition of Done - [x] Linked Gitea issue (above). - [x] Failing test committed before the implementation — twice: the Caddyfile contract test before the Caddyfiles, `make k8s-lint` before the chart. - [x] Implementation makes the test pass. - [x] Conventional Commits referencing the issues (`refs #25` / `refs #166`). - [ ] CI green — awaiting the run on this PR (`make k8s-lint`, `dotnet format` and the new unit self-check pass locally; the compose e2e and mutation lanes are CI's). - [ ] `docker compose up` from a fresh clone reaches green health checks within 3 minutes — the portal images were rebuilt and verified standalone, but a full `make up` run has not been done on this branch. Please confirm in review or let CI's smoke test speak. - [x] Docs updated — `docs/runbooks/kubernetes-talos.md` (new), `frontend-decisions.md`, `demo-script.md`, and the docs that named nginx. - [x] ADR added — ADR-0033 (chart) and ADR-0034 (Caddy). - [ ] Demo note in `docs/demo-script.md` — not added: the deployment target is not a user-visible slice, and the Caddy swap is invisible to the demo script beyond the wording fix included here. ## How it was verified Brought up from scratch on a single-node Talos v1.14.0 VM (6 vCPU / 10 GB, virtio disk) under virt-manager: **29 pods ready and four bootstrap Jobs complete in under three minutes, zero restarts**, using ~4.4 GB of the VM's 10 GB. - Full Common Ground path: portal Caddy → BFF → domain → Flowable → ACL → OpenZaak + Objecten → NRC → event-subscriber → projection → public register (`INGEDIEND`, reference matching the submitted registration). - Werkbak read with an MFA'd medewerker token → 200. - The browser flow driven with Playwright against `http://localhost:30140`: secure context, `crypto.subtle` present, Keycloak form reached, login completed, **no console errors**. - Routing checked against a stub BFF: SPA fallback serves deep links, each portal proxies its own groups, and a portal does *not* proxy a neighbour's group. ## Notes for reviewers Three bugs this shook out, each fixed at the cause rather than the symptom: 1. **`command` vs `args`.** Compose's `command:` replaces the image CMD; Kubernetes' replaces the ENTRYPOINT. Transcribing one to the other broke every upstream image that relies on its entrypoint — postgres refused to run as root, Keycloak tried to exec `start-dev`. The chart now `fail`s at render time on `command`. 2. **Concurrent migrations.** Both `/setup_configuration.sh` and `/start.sh` run `manage.py migrate`; compose serialises them with `depends_on`, Kubernetes has no such edge, so the init Job and its web pod raced (`relation "zgw_consumers_service" already exists`). The four Django services now do both steps in order in the web pod — which also deletes four workloads. 3. **`emptyDir` databases are wiped by any pod-template change.** `make k8s-reseed` now also restarts `event-subscriber` and `projection-api`, which create the projection schema on start and otherwise keep writing to a schema-less database. Known gaps / follow-ups: - **Secrets.** `values.yaml` carries the dev credentials in plain text (`admin/admin`, the ZGW client secret, the two Objecten tokens) and the chart has no `Secret` objects. Fine for a laptop demo, and exactly what #25's "production posture" ADR should address — I suggest a follow-up issue rather than stretching this PR. - **No CI gate for the chart yet.** `make k8s-lint` exists but is not wired into `.gitea/workflows/ci.yaml`, and nothing enforces that the chart and the compose file stay in step. Worth a small follow-up. - **This is two slices in one PR.** They were built and verified together and the diff is entangled (the chart was written against Caddy from the start), so splitting now would mean re-creating an nginx-shaped chart to throw away. Happy to split if you'd rather. - **Rebased onto #161** (merged as #165) rather than merged, to keep the history linear. One conflict, in the `unit:` target where both branches add a self-check line — resolved by keeping both. #161's `infra/host-browser.yml` arrived with `/usr/share/nginx/html/config.json` and is fixed to `/usr/share/caddy/` inside the `feat(portals)` commit, so no commit on this branch leaves that overlay pointing at a path the images no longer have.Reviewed-on: #167
143 lines
4.4 KiB
Smarty
143 lines
4.4 KiB
Smarty
{{/*
|
|
One pod spec for every workload, Deployment and Job alike. The chart is
|
|
values-driven on purpose: `.Values.workloads` is a near-literal transcription of
|
|
infra/docker-compose.yml, so the two stacks can be diffed by eye instead of by
|
|
archaeology. Adding a service is a values edit, not a template edit.
|
|
|
|
Called as: include "big.podspec" (dict "root" $ "name" $name "w" $w)
|
|
*/}}
|
|
{{- define "big.podspec" -}}
|
|
{{- $root := .root -}}
|
|
{{- $name := .name -}}
|
|
{{- $w := .w -}}
|
|
{{- with $root.Values.imagePullSecrets }}
|
|
imagePullSecrets:
|
|
{{- toYaml . | nindent 2 }}
|
|
{{- end }}
|
|
{{- with $w.waitFor }}
|
|
initContainers:
|
|
- name: wait-for-deps
|
|
image: {{ $root.Values.images.busybox }}
|
|
command:
|
|
- sh
|
|
- -c
|
|
- |
|
|
for t in {{ join " " . }}; do
|
|
echo "waiting for $t"
|
|
until nc -z "${t%:*}" "${t#*:}"; do sleep 2; done
|
|
done
|
|
{{- end }}
|
|
containers:
|
|
- name: {{ $name }}
|
|
image: {{ include "big.image" (dict "root" $root "name" $name "w" $w) }}
|
|
# Only this repo's images get the configured policy: their `dev` tag is mutable.
|
|
# Upstream tags are pinned, so IfNotPresent keeps them out of pod-template diffs —
|
|
# which matters because a changed template makes a Job unpatchable (immutable).
|
|
imagePullPolicy: {{ if $w.own }}{{ $root.Values.images.pullPolicy }}{{ else }}IfNotPresent{{ end }}
|
|
{{- if $w.command }}
|
|
{{- fail (printf "workload %s: use `args`, not `command` — compose's `command:` replaces CMD, but Kubernetes' `command:` replaces the image ENTRYPOINT (postgres would run as root, keycloak would exec `start-dev`)" $name) }}
|
|
{{- end }}
|
|
{{- with $w.args }}
|
|
args:
|
|
{{- toYaml . | nindent 6 }}
|
|
{{- end }}
|
|
{{- with $w.envFrom }}
|
|
envFrom:
|
|
{{- range . }}
|
|
- configMapRef:
|
|
# optional: an env group whose feature is disabled (e.g. otel) simply
|
|
# isn't rendered, and the pod must still start.
|
|
name: {{ printf "%s-env" . }}
|
|
optional: true
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- with $w.env }}
|
|
env:
|
|
{{- include "big.env" (list $root .) | nindent 6 }}
|
|
{{- end }}
|
|
{{- with $w.ports }}
|
|
ports:
|
|
{{- range . }}
|
|
- name: {{ .name }}
|
|
containerPort: {{ .targetPort | default .port }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- with $w.probe }}
|
|
readinessProbe:
|
|
{{- toYaml . | nindent 6 }}
|
|
{{- end }}
|
|
{{- with $w.resources }}
|
|
resources:
|
|
{{- toYaml . | nindent 6 }}
|
|
{{- end }}
|
|
{{- if or $w.files $w.data }}
|
|
volumeMounts:
|
|
{{- range $w.files }}
|
|
- name: {{ .configMap }}
|
|
mountPath: {{ .mountPath }}
|
|
{{- with .subPath }}
|
|
subPath: {{ . }}
|
|
{{- end }}
|
|
readOnly: true
|
|
{{- end }}
|
|
{{- with $w.data }}
|
|
- name: data
|
|
mountPath: {{ .mountPath }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- if or $w.files $w.data }}
|
|
volumes:
|
|
{{- range $w.files }}
|
|
- name: {{ .configMap }}
|
|
configMap:
|
|
name: {{ .configMap }}
|
|
{{- with .defaultMode }}
|
|
defaultMode: {{ . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- with $w.data }}
|
|
- name: data
|
|
{{- if $root.Values.persistence.storageClass }}
|
|
persistentVolumeClaim:
|
|
claimName: {{ $name }}-data
|
|
{{- else }}
|
|
# No StorageClass configured: the databases are emptyDir, so the stack needs
|
|
# no CSI driver to come up. Data then lives as long as the pod does — see
|
|
# docs/runbooks/kubernetes-talos.md for switching on local-path.
|
|
emptyDir: {}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end -}}
|
|
|
|
{{/* Image ref: `own: true` workloads are built from this repo, everything else is upstream. */}}
|
|
{{- define "big.image" -}}
|
|
{{- $root := .root -}}
|
|
{{- $w := .w -}}
|
|
{{- if $w.own -}}
|
|
{{- $ref := printf "%s/%s:%s" $root.Values.images.repositoryPrefix .name $root.Values.images.tag -}}
|
|
{{- with $root.Values.images.registry }}{{ printf "%s/%s" . $ref }}{{ else }}{{ $ref }}{{ end }}
|
|
{{- else -}}
|
|
{{- $w.image -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Env list from a map. Every value is run through `tpl`, so values.yaml can name
|
|
cluster-internal hosts ({{ .Release.Namespace }}) and the node address
|
|
({{ .Values.host }}) without the chart hard-coding either.
|
|
*/}}
|
|
{{- define "big.env" -}}
|
|
{{- $root := index . 0 -}}
|
|
{{- range $k, $v := index . 1 }}
|
|
- name: {{ $k }}
|
|
value: {{ tpl (toString $v) $root | quote }}
|
|
{{- end }}
|
|
{{- end -}}
|
|
|
|
{{- define "big.labels" -}}
|
|
app.kubernetes.io/name: {{ .name }}
|
|
app.kubernetes.io/instance: {{ .root.Release.Name }}
|
|
app.kubernetes.io/managed-by: Helm
|
|
{{- end -}}
|