refactor(infra): use upstream images verbatim, seed config via docker cp (refs #30)
Drops the inline-build images for the upstream services. The compose now references the published images directly (openzaak/open-zaak, openzaak/open-notificaties, keycloak, curl, flowable-rest) with no build for them, and the config they need is streamed into external named volumes by infra/seed-config.sh: rr-oz-config -> oz-init /app/setup_configuration (data.yaml) rr-kc-realms -> keycloak /opt/keycloak/data/import (realm exports) rr-fl-bpmn -> flowable-init /work (registratie.bpmn) How: the seeder creates each volume, `docker create`s a throwaway helper that mounts it, `docker cp`s the files in, and removes it. docker cp streams over the Docker API, so it works in Docker-in-Docker (the CI runner) where bind mounts mount empty. It uses plain `docker create`/`cp` — NOT `docker compose create`, which podman-compose (local dev) lacks. `external: true` fixed names keep the volumes identical across docker compose and podman-compose. Consequence: bare `docker compose up` no longer self-seeds, so use `make up` (seeds then starts). Every `*-up` target seeds first; `*-down` removes the external volume. acl/bff are still built (they're our apps, not upstream images). Verified end-to-end on podman-compose: `make keycloak-up` seeds rr-kc-realms, the upstream Keycloak mounts it, and --import-realm imports all four realms (digid realm returns 200). Seeder runs in ~2s. Docs updated: gitea-actions-gotchas.md, ci.md, openzaak.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -16,16 +16,18 @@ and CI cannot drift:
|
||||
| `lint` | `make lint` → `dotnet format … --verify-no-changes` | .NET 10 SDK |
|
||||
| `build` | `make build` → `dotnet build … -c Release` | .NET 10 SDK |
|
||||
| `unit` | `make unit` → `dotnet test … -c Release` | .NET 10 SDK |
|
||||
| `compose-smoke` | `make smoke` → `up -d` (full stack) → `up --wait` durable services → `down` | container engine + compose v2 |
|
||||
| `compose-smoke` | `make smoke` → seed config volumes → `up -d` (full stack) → `up --wait` durable services → `down` | container engine + compose v2 |
|
||||
|
||||
All `uses:` references are absolute, tag-pinned URLs (`https://github.com/actions/checkout@v4`,
|
||||
`https://github.com/actions/setup-dotnet@v4`) per CLAUDE.md §8.7 and §15 — Gitea
|
||||
Actions resolves them from GitHub.
|
||||
|
||||
> **`compose-smoke` runs on a containerized runner.** Workspace bind mounts do
|
||||
> **not** reach the sibling containers Compose starts, so config/assets are baked
|
||||
> into derived images instead of being mounted. If you add a service that needs a
|
||||
> repo file at runtime, bake it — don't bind-mount it. See
|
||||
> **not** reach the sibling containers Compose starts, so config/assets are
|
||||
> streamed into external named volumes via `docker cp` (`infra/seed-config.sh`),
|
||||
> and the upstream images are used verbatim (no build). If you add a service that
|
||||
> needs a repo file at runtime, seed it the same way — don't bind-mount it. Note:
|
||||
> bare `docker compose up` no longer self-seeds; use `make up`. See
|
||||
> [gitea-actions-gotchas.md](gitea-actions-gotchas.md).
|
||||
|
||||
## Running CI locally (`make ci`)
|
||||
|
||||
@@ -38,39 +38,50 @@ not happen on a runner that executes jobs directly on the host (the previous
|
||||
self-hosted `respellion-linux` setup), which is why switching to `ubuntu-latest`
|
||||
exposed it.
|
||||
|
||||
**Fix: bake assets into derived images instead of bind-mounting them.** Anything
|
||||
a Compose service needs at runtime that lives in the repo is `COPY`-ed into a
|
||||
small derived image, so it is present regardless of where the daemon runs. We use
|
||||
**`build.dockerfile_inline`** — the 2-line recipe lives in the compose file, so
|
||||
there are no standalone `Dockerfile` artifacts to maintain:
|
||||
**Fix: the upstream images are used verbatim (no build); config is streamed into
|
||||
external named volumes with `docker cp`.** `infra/seed-config.sh` creates a fixed-
|
||||
name volume per asset, runs a throwaway helper container that mounts it, and
|
||||
`docker cp`s the files in. `docker cp` streams bytes over the Docker API, so it
|
||||
works no matter where the daemon runs (including Docker-in-Docker). The services
|
||||
then mount those volumes:
|
||||
|
||||
| Asset | Derived image | Built by |
|
||||
| Asset | External volume | Mounted by → at |
|
||||
|---|---|---|
|
||||
| OpenZaak `setup_configuration/data.yaml` | `register-referentie/openzaak:dev` | `oz-init` `dockerfile_inline` |
|
||||
| Keycloak realm exports | `register-referentie/keycloak:dev` | `keycloak` `dockerfile_inline` |
|
||||
| `workflows/registratie.bpmn` | `register-referentie/flowable-init:dev` | `flowable-init` `dockerfile_inline` |
|
||||
| OpenZaak `setup_configuration/data.yaml` | `rr-oz-config` | `oz-init` → `/app/setup_configuration` |
|
||||
| Keycloak realm exports | `rr-kc-realms` | `keycloak` → `/opt/keycloak/data/import` |
|
||||
| `workflows/registratie.bpmn` | `rr-fl-bpmn` | `flowable-init` → `/work` |
|
||||
|
||||
The base image tag is interpolated by Compose (`${OPENZAAK_TAG}`) so pinning is
|
||||
unchanged. OpenZaak's `openzaak`/`oz-celery` reuse the one image `oz-init` builds.
|
||||
Open Notificaties needs **no** bake — `nrc-init` runs migrations only, so all NRC
|
||||
services use the plain base image.
|
||||
The volumes are declared `external: true` with fixed `name:`s so they resolve
|
||||
identically under docker compose and podman-compose. The seed step (`make` runs
|
||||
it before every `up`) recreates them fresh each time; `make down` / the
|
||||
per-service `*-down` targets remove them. Open Notificaties needs no config at all
|
||||
— `nrc-init` runs migrations only.
|
||||
|
||||
`dockerfile_inline` works on both the CI runner (docker compose) and local
|
||||
**podman-compose**, unlike `docker cp`-into-volumes (which needs `docker compose
|
||||
create`, a subcommand podman-compose lacks).
|
||||
**Two hard constraints drove this design:**
|
||||
|
||||
- Use plain `docker volume create` / `docker run` / `docker cp` — **not**
|
||||
`docker compose create`, which **podman-compose** (the local dev runtime) does
|
||||
not implement.
|
||||
- `docker cp` rather than a bind mount of the source dir, because that bind mount
|
||||
is exactly what fails on the containerized runner.
|
||||
|
||||
**Consequence: bare `docker compose up` no longer self-seeds** — the `external`
|
||||
volumes must be populated first, so use **`make up`** (or `make <svc>-up`), which
|
||||
seeds then starts. CI uses `make smoke`, which does the same.
|
||||
|
||||
**Why not the alternatives.**
|
||||
|
||||
- *Bake into a (possibly inline) image* — clean and portable, but it's a build;
|
||||
rejected here because the goal was to use the upstream images verbatim.
|
||||
- *Compose `configs:` with inline `content`* — Compose materialises these as a
|
||||
temp file on the **client** side and bind-mounts it, so it hits the exact same
|
||||
daemon-can't-see-the-path problem.
|
||||
- *A self-hosted runner that runs jobs on the host* — works, but reintroduces a
|
||||
bespoke runner and undoes the move to the hosted `ubuntu-latest` label.
|
||||
temp file on the **client** side and bind-mounts it → same daemon-can't-see-it
|
||||
problem.
|
||||
- *A self-hosted runner that runs jobs on the host* — bind mounts would then work
|
||||
with zero seeding, but it reintroduces a bespoke runner and undoes the move to
|
||||
the hosted `ubuntu-latest` label.
|
||||
|
||||
**Consequence for local dev.** There is now no bind mount of these config files,
|
||||
so the SELinux `:z`/`:Z` relabel flag is no longer needed anywhere in `infra/`,
|
||||
and rootless Podman no longer needs the files to be world-readable. One mechanism
|
||||
(build) works on both Podman locally and Docker-in-Docker in CI.
|
||||
No bind mounts of these config files remain, so the SELinux `:z`/`:Z` relabel flag
|
||||
is no longer needed anywhere in `infra/` (named volumes don't need relabeling).
|
||||
|
||||
## `--wait` fails on one-shot containers with no dependant
|
||||
|
||||
|
||||
@@ -73,7 +73,10 @@ The Makefile auto-points `DOCKER_HOST` at the Podman socket when it exists, so t
|
||||
resultaattypen — beyond the lean seed). List with `?status=alles`.
|
||||
- **Image tag.** Pinned to `openzaak/open-zaak:1.28.2` via `${OPENZAAK_TAG}` (bump
|
||||
deliberately, not via `:latest`).
|
||||
- **Config is baked, not mounted.** `setup_configuration/data.yaml` is `COPY`-ed into
|
||||
a derived image via `oz-init`'s inline Dockerfile (`build.dockerfile_inline` in
|
||||
the compose) rather than bind-mounted, so the init container finds it on Gitea's
|
||||
containerized CI runner too. See [gitea-actions-gotchas.md](gitea-actions-gotchas.md).
|
||||
- **Config arrives via a volume, not a bind mount.** `setup_configuration/data.yaml`
|
||||
is streamed into the external `rr-oz-config` volume by `infra/seed-config.sh`
|
||||
(`docker cp`) and mounted at `/app/setup_configuration`, so the init container
|
||||
finds it on Gitea's containerized CI runner too (bind mounts don't reach sibling
|
||||
containers there). The image is the upstream `openzaak/open-zaak` verbatim — no
|
||||
build. Run via `make openzaak-up` (seeds first). See
|
||||
[gitea-actions-gotchas.md](gitea-actions-gotchas.md).
|
||||
|
||||
Reference in New Issue
Block a user