Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  notandClaude Opus 5
|
dfa1a370ac | ||
|
notandClaude Opus 5
|
e8cb1ec7e9 | ||
|
notandClaude Opus 5
|
de6db7d35b | ||
|
notandClaude Opus 5
|
d17e79959b | ||
|
notandClaude Opus 5
|
fc036d53d5 |
@@ -27,13 +27,10 @@ jobs:
|
||||
# `kubectl port-forward` — runbook §5. Override with repo variables.
|
||||
TALOS_VM_IP: ${{ vars.TALOS_VM_IP }}
|
||||
TALOS_HOST: ${{ vars.TALOS_HOST }}
|
||||
# Set it when the labs Caddy publishes the portals: Keycloak's public https
|
||||
# origin, e.g. https://big-auth.labs.respellion.tech (runbook, "Publishing
|
||||
# through the labs Caddy").
|
||||
KEYCLOAK_URL: ${{ vars.KEYCLOAK_URL }}
|
||||
# `true` fills in the medewerker OTP step for the public demo (chart value
|
||||
# demo.otpAutofill). The fixture secret is committed: demo only.
|
||||
OTP_AUTOFILL: ${{ vars.OTP_AUTOFILL }}
|
||||
# Set it and the stack is published over TLS on <sub>.<domain> by the
|
||||
# in-cluster edge (ADR-0035, runbook §10). Empty = NodePorts, as before.
|
||||
PUBLIC_DOMAIN: ${{ vars.PUBLIC_DOMAIN }}
|
||||
PUBLIC_EMAIL: ${{ vars.PUBLIC_EMAIL }}
|
||||
steps:
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
|
||||
@@ -97,10 +94,12 @@ jobs:
|
||||
# Job template from wedging the upgrade (`cannot patch … with kind Job`).
|
||||
- name: Deploy the chart
|
||||
run: |
|
||||
set -euo pipefail
|
||||
publish="${PUBLIC_DOMAIN:+--set public.domain=$PUBLIC_DOMAIN --set public.email=${PUBLIC_EMAIL:-}}"
|
||||
make k8s-reseed \
|
||||
TALOS_HOST=${TALOS_HOST:-localhost} \
|
||||
K8S_REGISTRY=${TALOS_VM_IP:-192.168.122.173}:30500 \
|
||||
K8S_SET="${KEYCLOAK_URL:+--set keycloakUrl=$KEYCLOAK_URL} --set demo.otpAutofill=${OTP_AUTOFILL:-false}"
|
||||
K8S_SET="$publish"
|
||||
|
||||
# `dev` is a mutable tag and helm sees an unchanged pod template, so the
|
||||
# new images only land on a restart (pullPolicy is already Always).
|
||||
@@ -116,6 +115,12 @@ jobs:
|
||||
- name: Smoke the public register
|
||||
run: curl -fsS --retry 10 --retry-delay 6 --retry-all-errors http://localhost:30141/openbaar/register
|
||||
|
||||
- name: Pods on failure
|
||||
# Cluster-wide, not just `big`: the first thing that can fail is the registry
|
||||
# in its own namespace, and a scheduling problem shows up in the events, not
|
||||
# in `rollout status` — which only ever says "timed out waiting".
|
||||
- name: Pods and events on failure
|
||||
if: failure()
|
||||
run: kubectl -n big get pods,jobs || true
|
||||
run: |
|
||||
kubectl get pods -A -o wide || true
|
||||
kubectl -n big get jobs || true
|
||||
kubectl get events -A --sort-by=.lastTimestamp | tail -30 || true
|
||||
|
||||
@@ -366,6 +366,36 @@ immutable, so `helm upgrade` is rejected with `cannot patch "…" with kind Job`
|
||||
every time a PR is squash-merged to `main` (and on demand via *Run workflow*). PR CI is the
|
||||
merge gate, so the workflow deploys without re-running the checks.
|
||||
|
||||
**Prerequisite: the VM must have been installed with the §1 patch.** A stock Talos config
|
||||
gives you a node that still carries the control-plane taint and knows nothing about the
|
||||
plain-HTTP registry, and the deploy hits those in that order: the `registry` pod sits
|
||||
`Pending` until `rollout status` times out, and once that is fixed every repo image fails to
|
||||
pull. Two separate fixes:
|
||||
|
||||
```bash
|
||||
# on the Fedora host — 1. let workloads onto the only node (§1)
|
||||
export KUBECONFIG=~/talos-kubeconfig-local
|
||||
kubectl taint node --all node-role.kubernetes.io/control-plane-
|
||||
|
||||
# 2. trust the in-cluster registry over plain HTTP (§2)
|
||||
cat > /tmp/registry-patch.yaml <<'YAML'
|
||||
machine:
|
||||
registries:
|
||||
mirrors:
|
||||
"<TALOS_VM_IP>:30500":
|
||||
endpoints:
|
||||
- http://<TALOS_VM_IP>:30500
|
||||
YAML
|
||||
talosctl -n <TALOS_VM_IP> -e <TALOS_VM_IP> patch mc --patch @/tmp/registry-patch.yaml
|
||||
```
|
||||
|
||||
Keep those two apart. On Talos 1.14 a patch that also sets
|
||||
`cluster.allowSchedulingOnControlPlanes` is rejected with *".cluster.allowSchedulingOnControlPlanes
|
||||
is already set in v1alpha1 config"* — the field moved out of the v1alpha1 schema, the same way
|
||||
`machine.install` did (§1) — and the rejection takes the whole patch with it, so the mirror
|
||||
silently doesn't land either. `kubectl taint` is the documented way (§1); it is undone if the
|
||||
node ever re-registers, which is a reboot, not a deploy.
|
||||
|
||||
The cluster's API and registry are not exposed publicly, so the job forwards them over the
|
||||
same SSH hop the Gitea-runner pipeline uses:
|
||||
|
||||
@@ -397,58 +427,30 @@ Settings, all on the repository in Gitea:
|
||||
The last step smokes `GET /openbaar/register` through the openbaar portal, which exercises
|
||||
portal → Caddy → BFF → projection. An empty register passes; a 502 does not.
|
||||
|
||||
### Reaching the portals from a laptop
|
||||
|
||||
The deployed portals are pinned to `http://localhost:30180` for Keycloak (§5), so a browser
|
||||
needs **all five** browser-facing ports on its own localhost — the portal alone is not
|
||||
enough, and a missing Keycloak shows up as `ERR_CONNECTION_REFUSED` on
|
||||
`/realms/*/.well-known/openid-configuration` followed by an opaque `ERROR Error: [object Object]`.
|
||||
`make k8s-portals` does this when kubectl can reach the cluster; through the lab server one
|
||||
SSH does it without a kubeconfig at all:
|
||||
|
||||
```bash
|
||||
ssh -N -p 6667 \
|
||||
-L 30140:<TALOS_VM_IP>:30140 -L 30141:<TALOS_VM_IP>:30141 \
|
||||
-L 30142:<TALOS_VM_IP>:30142 -L 30143:<TALOS_VM_IP>:30143 \
|
||||
-L 30180:<TALOS_VM_IP>:30180 \
|
||||
user@labs.respellion.tech
|
||||
```
|
||||
|
||||
Then the §5 table's URLs work as written. The admin UIs (OpenZaak, Flowable, …) need no
|
||||
forward — they are server-rendered, so the VM's address is fine.
|
||||
|
||||
Not covered: the portals still need `make k8s-portals` (or an SSH forward) to be usable in a
|
||||
browser, because PKCE needs a secure context (§5). Giving the server a hostname + TLS is the
|
||||
upgrade path.
|
||||
|
||||
## Publishing through the labs Caddy
|
||||
|
||||
The portals can be reached on real hostnames through the Caddy that already fronts
|
||||
`*.labs.respellion.tech` (repo `Infra`, `infra/development/`). The chain:
|
||||
|
||||
```
|
||||
browser → Caddy (labs server, TLS) → openssh-server:3014x/30180
|
||||
→ reverse SSH tunnel → Fedora host → <TALOS_VM_IP>:3014x/30180 (NodePorts)
|
||||
```
|
||||
|
||||
| URL | NodePort |
|
||||
|---|---|
|
||||
| `https://big-register.labs.respellion.tech` | 30141 openbaar |
|
||||
| `https://big-mijn.labs.respellion.tech` | 30140 self-service |
|
||||
| `https://big-behandel.labs.respellion.tech` | 30142 behandel |
|
||||
| `https://big-beheer.labs.respellion.tech` | 30143 beheer |
|
||||
| `https://big-auth.labs.respellion.tech` | 30180 Keycloak (`/admin` blocked) |
|
||||
|
||||
HTTPS makes the portals a secure context, so PKCE works without port-forwards — but
|
||||
Keycloak's issuer must be the public origin. Deploy with it:
|
||||
|
||||
```bash
|
||||
make k8s-up TALOS_HOST=localhost K8S_REGISTRY=<TALOS_HOST>:30500 \
|
||||
K8S_SET="--set keycloakUrl=https://big-auth.labs.respellion.tech"
|
||||
```
|
||||
|
||||
For deploy-on-merge, set the repository variable `KEYCLOAK_URL` to the same value.
|
||||
With it set, the `localhost` port-forwards (§5) no longer log in: the issuer is one string.
|
||||
|
||||
Staff logins still hit the enforced OTP step. For a demo, set the repository variable
|
||||
`OTP_AUTOFILL=true` (chart value `demo.otpAutofill`): Keycloak then uses the `big-demo`
|
||||
theme, which fills in and submits the code from the fixture secret, so the step is visible
|
||||
but needs no authenticator. Keycloak restarts when the value flips. Demo only — the secret
|
||||
is committed.
|
||||
|
||||
The theme lives in `infra/keycloak/themes/big-demo/` and is seeded as the `rr-kc-theme`
|
||||
ConfigMap by `infra/helm/seed-configmaps.sh` on every deploy. Keycloak runs `start-dev`,
|
||||
which doesn't cache themes, so an edit shows up about a minute after the ConfigMap changes.
|
||||
A *new* theme file also needs a key in the seed script and a path in the keycloak `files`
|
||||
in `values.yaml`.
|
||||
|
||||
One-time setup:
|
||||
|
||||
1. Fedora host: install `infra/development/big-portals-tunnel.service` from the Infra repo
|
||||
(instructions in the file).
|
||||
2. Labs server: deploy the Infra `Caddyfile` + `compose.yml` (Caddy joins the
|
||||
`openssh_default` network to reach the tunnel ends).
|
||||
|
||||
## What is not ported
|
||||
|
||||
- **Observability** (Tempo, Prometheus, Grafana) is defined but disabled — those are built
|
||||
|
||||
@@ -57,9 +57,6 @@ services:
|
||||
# share this anchor and ignore it — they don't run uwsgi.
|
||||
UWSGI_PROCESSES: "1"
|
||||
UWSGI_THREADS: "2"
|
||||
# Same lever for oz-celery: unset, the worker forks one process per CPU (22 on the lab node,
|
||||
# ~225 MB each), which OOM-killed the shared runner mid-verify-stack. Only celery reads it.
|
||||
CELERY_WORKER_CONCURRENCY: "2"
|
||||
DJANGO_SETTINGS_MODULE: openzaak.conf.docker
|
||||
SECRET_KEY: ${OZ_SECRET_KEY:-dev-only-not-for-production}
|
||||
DB_HOST: oz-db
|
||||
@@ -147,8 +144,6 @@ services:
|
||||
# 1 uWSGI worker, not the image default of 4×4 (#147) — see the oz-env note above.
|
||||
UWSGI_PROCESSES: "1"
|
||||
UWSGI_THREADS: "2"
|
||||
# Two celery workers, not one per CPU — see the oz-env note above.
|
||||
CELERY_WORKER_CONCURRENCY: "2"
|
||||
DJANGO_SETTINGS_MODULE: nrc.conf.docker
|
||||
SECRET_KEY: ${NRC_SECRET_KEY:-dev-only-not-for-production}
|
||||
DB_HOST: nrc-db
|
||||
|
||||
@@ -94,10 +94,6 @@ volumes:
|
||||
{{- with .defaultMode }}
|
||||
defaultMode: {{ . }}
|
||||
{{- end }}
|
||||
{{- with .items }}
|
||||
items:
|
||||
{{- toYaml . | nindent 8 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- with $w.data }}
|
||||
- name: data
|
||||
@@ -129,26 +125,14 @@ volumes:
|
||||
{{/*
|
||||
Env list from a map. Every value is run through `tpl`, so values.yaml can name
|
||||
cluster-internal hosts ({{ .Release.Namespace }}) and the node address
|
||||
({{ .Values.host }}) without the chart hard-coding either. A value that renders
|
||||
empty is left out, which is how a setting is made conditional on a chart value.
|
||||
({{ .Values.host }}) without the chart hard-coding either.
|
||||
*/}}
|
||||
{{- define "big.env" -}}
|
||||
{{- $root := index . 0 -}}
|
||||
{{- range $k, $v := index . 1 }}
|
||||
{{- $val := tpl (toString $v) $root }}
|
||||
{{- if $val }}
|
||||
- name: {{ $k }}
|
||||
value: {{ $val | quote }}
|
||||
value: {{ tpl (toString $v) $root | quote }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end -}}
|
||||
|
||||
{{/*
|
||||
The origin a browser reaches Keycloak on: the issuer Keycloak pins and the
|
||||
authority the portals use, from one place so they cannot drift (ADR-0010).
|
||||
*/}}
|
||||
{{- define "big.keycloakUrl" -}}
|
||||
{{- .Values.keycloakUrl | default (printf "http://%s:%v" .Values.host (index .Values.nodePorts "keycloak")) -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "big.labels" -}}
|
||||
|
||||
@@ -40,5 +40,5 @@ metadata:
|
||||
{{- include "big.labels" (dict "root" $ "name" (printf "portal-config-%s" $realm)) | nindent 4 }}
|
||||
data:
|
||||
config.json: |
|
||||
{ "authority": "{{ include "big.keycloakUrl" $ }}/realms/{{ $realm }}" }
|
||||
{ "authority": "{{ printf "http://%s:%v" $.Values.host (index $.Values.nodePorts "keycloak") }}/realms/{{ $realm }}" }
|
||||
{{- end }}
|
||||
|
||||
@@ -28,7 +28,7 @@ spec:
|
||||
{{- range $w.files }}
|
||||
{{- if hasPrefix "portal-config-" .configMap }}
|
||||
annotations:
|
||||
checksum/portal-config: {{ include "big.keycloakUrl" $ | sha256sum }}
|
||||
checksum/portal-config: {{ printf "%s|%v" $.Values.host (index $.Values.nodePorts "keycloak") | sha256sum }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
labels:
|
||||
|
||||
@@ -25,17 +25,6 @@
|
||||
# string, so browser tokens and the BFF's discovered issuer agree.
|
||||
host: 192.168.122.100
|
||||
|
||||
# Set when a TLS proxy outside the cluster publishes Keycloak: the full origin, no
|
||||
# trailing slash. It replaces `host` + Keycloak's NodePort as the issuer and the
|
||||
# portals' authority (runbook, "Publishing through the labs Caddy").
|
||||
keycloakUrl: ""
|
||||
|
||||
demo:
|
||||
# Fill in and submit the medewerker OTP step from the fixture secret, so a public
|
||||
# demo shows MFA enforced without an authenticator: makes the big-demo theme
|
||||
# (infra/keycloak/themes/big-demo) Keycloak's default. Demo only: the secret is committed.
|
||||
otpAutofill: false
|
||||
|
||||
# Set when pulling from a private registry (e.g. the Gitea Container Registry).
|
||||
imagePullSecrets: []
|
||||
|
||||
@@ -82,7 +71,6 @@ envGroups:
|
||||
oz:
|
||||
UWSGI_PROCESSES: "1"
|
||||
UWSGI_THREADS: "2"
|
||||
CELERY_WORKER_CONCURRENCY: "2"
|
||||
DJANGO_SETTINGS_MODULE: openzaak.conf.docker
|
||||
SECRET_KEY: dev-only-not-for-production
|
||||
DB_HOST: oz-db
|
||||
@@ -105,7 +93,6 @@ envGroups:
|
||||
nrc:
|
||||
UWSGI_PROCESSES: "1"
|
||||
UWSGI_THREADS: "2"
|
||||
CELERY_WORKER_CONCURRENCY: "2"
|
||||
DJANGO_SETTINGS_MODULE: nrc.conf.docker
|
||||
SECRET_KEY: dev-only-not-for-production
|
||||
DB_HOST: nrc-db
|
||||
@@ -281,31 +268,13 @@ workloads:
|
||||
# Pin the issuer to the address the browser uses, and let backchannel calls
|
||||
# keep using keycloak:8080 — the BFF discovers metadata in-cluster and gets
|
||||
# this issuer back, which is what browser tokens carry (infra/host-browser.yml).
|
||||
KC_HOSTNAME: '{{ include "big.keycloakUrl" . }}'
|
||||
KC_HOSTNAME: "http://{{ .Values.host }}:{{ index .Values.nodePorts \"keycloak\" }}"
|
||||
KC_HOSTNAME_BACKCHANNEL_DYNAMIC: "true"
|
||||
# Only rendered with demo.otpAutofill (big.env skips empty values); off, Keycloak
|
||||
# keeps its stock theme and the mounted big-demo theme is unused.
|
||||
KC_SPI_THEME_DEFAULT: '{{ if .Values.demo.otpAutofill }}big-demo{{ end }}'
|
||||
# Behind a TLS proxy (keycloakUrl) the dynamic backchannel URLs — token,
|
||||
# userinfo, certs — take their scheme from the request, which reaches Keycloak
|
||||
# as plain http; trusting X-Forwarded-Proto keeps them https so the browser
|
||||
# doesn't block them as mixed content. In-cluster calls send no such header.
|
||||
KC_PROXY_HEADERS: xforwarded
|
||||
ports: [{ name: http, port: 8080 }]
|
||||
# TCP, not /health/ready on the management port: nothing here gates on realm
|
||||
# import, and a wrong health path would leave the Service with no endpoints.
|
||||
probe: { tcpSocket: { port: 8080 }, initialDelaySeconds: 15 }
|
||||
files:
|
||||
- { configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }
|
||||
# infra/keycloak/themes/big-demo, seeded by infra/helm/seed-configmaps.sh.
|
||||
- configMap: rr-kc-theme
|
||||
mountPath: /opt/keycloak/themes/big-demo
|
||||
items:
|
||||
- { key: login.properties, path: login/theme.properties }
|
||||
- { key: otp-autofill.js, path: login/resources/js/otp-autofill.js }
|
||||
- { key: account.properties, path: account/theme.properties }
|
||||
- { key: admin.properties, path: admin/theme.properties }
|
||||
- { key: email.properties, path: email/theme.properties }
|
||||
files: [{ configMap: rr-kc-realms, mountPath: /opt/keycloak/data/import }]
|
||||
|
||||
# ── Flowable (S-03) ─────────────────────────────────────────────────────────
|
||||
flowable-db:
|
||||
|
||||
@@ -30,15 +30,6 @@ seed() { # name <kubectl --from-file args...>
|
||||
seed rr-oz-config --from-file="$repo/infra/openzaak/setup_configuration/"
|
||||
seed rr-nrc-config --from-file="$repo/infra/opennotificaties/setup_configuration/"
|
||||
seed rr-kc-realms --from-file="$repo/infra/keycloak/realms/"
|
||||
# The big-demo login theme (demo.otpAutofill). ConfigMap keys are flat, so each
|
||||
# file gets a key here and its path back in the keycloak `files` in values.yaml.
|
||||
theme="$repo/infra/keycloak/themes/big-demo"
|
||||
seed rr-kc-theme \
|
||||
--from-file=login.properties="$theme/login/theme.properties" \
|
||||
--from-file=otp-autofill.js="$theme/login/resources/js/otp-autofill.js" \
|
||||
--from-file=account.properties="$theme/account/theme.properties" \
|
||||
--from-file=admin.properties="$theme/admin/theme.properties" \
|
||||
--from-file=email.properties="$theme/email/theme.properties"
|
||||
seed rr-objecttypen-config --from-file="$repo/infra/objecttypen/setup_configuration/"
|
||||
seed rr-objecten-config --from-file="$repo/infra/objecten/setup_configuration/"
|
||||
# register.py + the RegisterRecord JSON schema (the __pycache__ dir is skipped:
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the account page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak.v3
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the admin page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak.v2
|
||||
@@ -1,4 +0,0 @@
|
||||
# The chart makes big-demo the default for every theme type, and Keycloak does not
|
||||
# fall back for a type a theme lacks (the email page then fails), so each type is
|
||||
# declared as a plain child of Keycloak 26's own default.
|
||||
parent=keycloak
|
||||
@@ -1,24 +0,0 @@
|
||||
// RFC 6238 with Keycloak's default policy (HmacSHA1, 6 digits, 30 s) over the
|
||||
// raw bytes of the medewerker fixture secret — same as tests/e2e/keycloak-login.ts.
|
||||
document.addEventListener('DOMContentLoaded', async () => {
|
||||
const input = document.querySelector('input[name="otp"]');
|
||||
if (!input || !input.form) return;
|
||||
const key = await crypto.subtle.importKey('raw',
|
||||
new TextEncoder().encode('BIGMEDEWERKEROTPSEED'), { name: 'HMAC', hash: 'SHA-1' }, false, ['sign']);
|
||||
// A code is single-use, so a second login in the same window spends the next
|
||||
// counter (Keycloak's look-ahead accepts it). Past that, fill but don't submit,
|
||||
// so a rejected code can't turn into a submit loop.
|
||||
const now = Math.floor(Date.now() / 30000);
|
||||
let last = -1;
|
||||
try { last = Number(sessionStorage.getItem('big-otp-counter')) || -1; } catch {}
|
||||
const counter = Math.max(now, last + 1);
|
||||
const msg = new DataView(new ArrayBuffer(8));
|
||||
msg.setBigUint64(0, BigInt(counter));
|
||||
const mac = new Uint8Array(await crypto.subtle.sign('HMAC', key, msg.buffer));
|
||||
const o = mac[19] & 0x0f;
|
||||
const n = ((mac[o] & 0x7f) << 24 | mac[o + 1] << 16 | mac[o + 2] << 8 | mac[o + 3]) % 1e6;
|
||||
input.value = String(n).padStart(6, '0');
|
||||
if (counter > now + 1) return;
|
||||
try { sessionStorage.setItem('big-otp-counter', String(counter)); } catch {}
|
||||
input.form.requestSubmit();
|
||||
});
|
||||
@@ -1,11 +0,0 @@
|
||||
# Demo login theme for the public Talos deployment: keycloak.v2 plus a script that
|
||||
# fills in and submits the medewerker OTP step from the committed fixture secret
|
||||
# (docs/runbooks/keycloak.md). Only used when the chart's demo.otpAutofill is on —
|
||||
# it then becomes Keycloak's default theme. Never enable it anywhere real.
|
||||
#
|
||||
# Add styles, messages or template overrides here as in any Keycloak theme
|
||||
# (https://www.keycloak.org/ui-customization/themes); new files must also be
|
||||
# listed in infra/helm/seed-configmaps.sh and the keycloak `files` in values.yaml.
|
||||
parent=keycloak.v2
|
||||
import=common/keycloak
|
||||
scripts=js/otp-autofill.js
|
||||
Reference in New Issue
Block a user