## What & why Use the standard `$GITHUB_STEP_SUMMARY` (Gitea 1.27 + act_runner 2.0.0) to surface on the run page what was previously buried in logs or download-only artifacts. All five quick wins from #136, **reporting-only** — no job's pass/fail gating changes. Closes #136 ### Items 1. **Mutation scores** — added the `markdown` reporter to each `stryker-config.json`; the `mutation` job concatenates each service's `mutation-report.md` into the summary (`if: always()`). Also reveals where `make mutation` stopped on a ratchet break. 2. **Per-frontend tests** — the 4 apps' `test` targets emit vitest JSON to `test-output/{projectName}.json` (Nx token interpolation); `infra/vitest-summary.py` renders a per-frontend table. 3. **Per-service unit tests** — `make unit` now writes TRX; `infra/trx-summary.py` renders a per-service table (service name derived from the `services/<name>/` path, so `domain` shows, not `big.tests`). 4. **e2e per-spec results** — Playwright writes `playwright-report.json`; `run-e2e-check.sh` copies it out of the container (capturing the exit code first); `infra/playwright-summary.py` renders a per-spec table. Turns a red e2e into a one-glance "which spec". 5. **verify-stack check table** — each live-stack check has an `id`; a final `if: always()` step tabulates each check's ✅/❌/⏭️. Docs: `gitea-actions-gotchas.md` §8 (version requirement + `$GITHUB_STEP_SUMMARY` guard + step-level `always()` note). ### Notes - Every summary write is guarded with `[ -n "${GITHUB_STEP_SUMMARY:-}" ]`, so it no-ops on an unsupported runner / locally. - New helper scripts are stdlib-only Python, matching the existing `infra/*.py` check scripts (no new dependency — a few lines of parsing rather than a test-logger package). - `TestResults/` and `test-output/` gitignored. - This is also the first PR-run exercising the #135 verify-stack fix end to end. ## Verified locally `make unit` (TRX) ✓ · 4 apps' vitest JSON ✓ · ACL Stryker markdown report ✓ · all four parsers + the two summary shell blocks ✓ · `ci.yaml` + `run-e2e-check.sh` syntax ✓. The rendered summaries themselves only appear on the run page — this PR's CI run is the end-to-end check. ## Definition of Done - [x] Each item writes to `$GITHUB_STEP_SUMMARY` (guarded), renders on the run page. - [x] No change to any job's pass/fail gating. - [x] Conventional Commits referencing #136 (one per item + docs). - [ ] CI green; summaries visible on the run. - [x] Runbook note (gotchas §8). 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #137
This commit was merged in pull request #137.
This commit is contained in:
@@ -70,6 +70,12 @@ jobs:
|
||||
restore-keys: |
|
||||
nuget-${{ runner.os }}-
|
||||
- run: make unit
|
||||
# Job summary (#136): a per-service pass/fail table from the TRX `make unit` wrote.
|
||||
- name: Unit test summary
|
||||
if: always()
|
||||
run: |
|
||||
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
|
||||
python3 infra/trx-summary.py TestResults >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# Frontend (Nx/Angular) lane: install with pnpm, then Nx lint + test + build.
|
||||
frontend:
|
||||
@@ -84,6 +90,12 @@ jobs:
|
||||
node-version: '24'
|
||||
cache: 'pnpm'
|
||||
- run: make frontend
|
||||
# Job summary (#136): a per-frontend (app) pass/fail table from the vitest JSON each app wrote.
|
||||
- name: Frontend test summary
|
||||
if: always()
|
||||
run: |
|
||||
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
|
||||
python3 infra/vitest-summary.py test-output >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
mutation:
|
||||
runs-on: ubuntu-latest
|
||||
@@ -99,6 +111,29 @@ jobs:
|
||||
restore-keys: |
|
||||
nuget-${{ runner.os }}-
|
||||
- run: make mutation
|
||||
# Job summary (#136): render each service's Stryker Markdown report on the run page (Gitea
|
||||
# 1.27 $GITHUB_STEP_SUMMARY). `if: always()` so a ratchet break still reports — and because
|
||||
# `make mutation` stops at the first break, the summary also shows exactly where it stopped.
|
||||
# Guarded so it no-ops on a runner/server without summary support. Strips the report's UTF-8 BOM.
|
||||
- name: Mutation score summary
|
||||
if: always()
|
||||
run: |
|
||||
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
|
||||
{
|
||||
echo "## 🧬 Mutation testing"
|
||||
echo
|
||||
for svc in acl event-subscriber domain bff; do
|
||||
echo "### $svc"
|
||||
echo
|
||||
report=$(ls services/"$svc"/StrykerOutput/*/reports/mutation-report.md 2>/dev/null | sort | tail -1)
|
||||
if [ -n "$report" ]; then
|
||||
sed '1s/^\xef\xbb\xbf//' "$report"
|
||||
else
|
||||
echo "_No report — \`make mutation\` stopped before \`$svc\` (earlier ratchet break)._"
|
||||
fi
|
||||
echo
|
||||
done
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
# Publish the Stryker HTML reports. `if: always()` uploads them even when the
|
||||
# ratchet fails — that is exactly when you want to inspect the survivors.
|
||||
# `continue-on-error` keeps the upload best-effort: the mutation *gate* is the
|
||||
@@ -158,26 +193,80 @@ jobs:
|
||||
- uses: https://github.com/actions/checkout@v4
|
||||
# Bring the full stack up + wait for health — this also is the DoD "compose up
|
||||
# reaches green health" smoke (it replaces the old compose-smoke job).
|
||||
# Each check carries an `id` so the summary step below can report its per-check outcome (#136).
|
||||
# A failed check skips the rest (no step `if:`), so the table shows exactly where it stopped.
|
||||
- name: Bring up the full stack & wait for health
|
||||
id: up
|
||||
run: make verify-up
|
||||
- name: Observability backplane (Grafana + Tempo + Prometheus datasources)
|
||||
id: obs
|
||||
run: OBS_TIMEOUT=180 make verify-observability
|
||||
- name: ACL ↔ OpenZaak integration tests
|
||||
id: acl
|
||||
run: make verify-acl
|
||||
- name: OpenZaak → NRC notification delivery
|
||||
id: nrc
|
||||
run: make verify-nrc
|
||||
- name: OpenZaak → NRC → Event Subscriber → projection-api
|
||||
id: projection
|
||||
run: make verify-projection
|
||||
- name: Domain → Flowable → ACL → OpenZaak
|
||||
id: domain
|
||||
run: make verify-domain
|
||||
- name: BFF → Keycloak + domain + projection
|
||||
id: bff
|
||||
run: make verify-bff
|
||||
- name: Distributed traces reach Tempo (one connected trace across services)
|
||||
id: tracing
|
||||
run: TRACING_TIMEOUT=120 make verify-tracing
|
||||
- name: Golden-signal metrics scraped by Prometheus (/metrics on every service)
|
||||
id: metrics
|
||||
run: METRICS_TIMEOUT=120 make verify-metrics
|
||||
- name: Self-service e2e (Playwright, login → submit → success)
|
||||
id: e2e
|
||||
run: make verify-e2e
|
||||
# Job summary (#136): a pass/fail table of every live-stack check, so a red verify-stack shows
|
||||
# which check failed at a glance. `if: always()` (step-level — safe on runner 2.0.0, unlike the
|
||||
# job-level status-function `if` of #134) so it renders even after a check fails.
|
||||
- name: verify-stack check summary
|
||||
if: always()
|
||||
env:
|
||||
UP: ${{ steps.up.outcome }}
|
||||
OBS: ${{ steps.obs.outcome }}
|
||||
ACL: ${{ steps.acl.outcome }}
|
||||
NRC: ${{ steps.nrc.outcome }}
|
||||
PROJECTION: ${{ steps.projection.outcome }}
|
||||
DOMAIN: ${{ steps.domain.outcome }}
|
||||
BFF: ${{ steps.bff.outcome }}
|
||||
TRACING: ${{ steps.tracing.outcome }}
|
||||
METRICS: ${{ steps.metrics.outcome }}
|
||||
E2E: ${{ steps.e2e.outcome }}
|
||||
run: |
|
||||
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
|
||||
icon() { case "$1" in success) echo "✅";; failure) echo "❌";; skipped) echo "⏭️";; cancelled) echo "🚫";; *) echo "❔ ${1:-—}";; esac; }
|
||||
{
|
||||
echo "## 🔌 verify-stack checks"
|
||||
echo
|
||||
echo "| Check | Result |"
|
||||
echo "| ----- | :----: |"
|
||||
echo "| Bring up + health | $(icon "$UP") |"
|
||||
echo "| Observability backplane | $(icon "$OBS") |"
|
||||
echo "| ACL ↔ OpenZaak | $(icon "$ACL") |"
|
||||
echo "| OpenZaak → NRC | $(icon "$NRC") |"
|
||||
echo "| NRC → Event Subscriber → projection | $(icon "$PROJECTION") |"
|
||||
echo "| Domain → Flowable → ACL → OpenZaak | $(icon "$DOMAIN") |"
|
||||
echo "| BFF → Keycloak + domain + projection | $(icon "$BFF") |"
|
||||
echo "| Distributed traces (Tempo) | $(icon "$TRACING") |"
|
||||
echo "| Golden-signal metrics (Prometheus) | $(icon "$METRICS") |"
|
||||
echo "| Self-service e2e (Playwright) | $(icon "$E2E") |"
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
# Job summary (#136): per-spec Playwright results, from the JSON report run-e2e-check.sh copied
|
||||
# out of the e2e container. Turns a red e2e into a one-glance "which spec" instead of a log dive.
|
||||
- name: e2e spec summary
|
||||
if: always()
|
||||
run: |
|
||||
[ -n "${GITHUB_STEP_SUMMARY:-}" ] || exit 0
|
||||
python3 infra/playwright-summary.py tests/e2e/playwright-report.json >> "$GITHUB_STEP_SUMMARY"
|
||||
# Log dump must precede teardown (which removes the containers).
|
||||
- name: Dump container logs on failure
|
||||
if: failure()
|
||||
|
||||
Reference in New Issue
Block a user