## What & why After the Gitea 1.27 + act_runner 2.0.0 upgrade, `verify-stack` never starts: the run sits in `waiting` forever with no logs for that job, while the other five jobs pass — so `main` stays pending/red (P0). See #134. Closes #134 ### Root cause `verify-stack` was the only job gated by a status-function `if` on top of `needs`: ```yaml verify-stack: needs: [mutation] if: ${{ !cancelled() }} ``` Gitea 1.27 reworked cancellation/aggregation so that `always()`/`cancelled()`-gated `needs` jobs route through a new transitional **`Cancelling`** state + server↔runner **capability negotiation** ("Requires Gitea Runner 2.0.0"). On this 1.27 + 2.0.0 pairing that handshake doesn't resolve, so the job is never dispatched and never leaves `waiting`. Plain jobs (no `if`/`needs`) are unaffected — exactly the observed pattern. It worked pre-upgrade (old runner). ### Fix Drop the `if: ${{ !cancelled() }}`; keep `needs: [mutation]`. Default `if: success()` dispatches normally and still serialises the two memory-heavy jobs (OOM avoidance, #126). **Trade-off:** the `!cancelled()` (added in #127) let verify-stack run even when the mutation ratchet fails. Now a failing mutation skips verify-stack; the fix-and-re-push re-run exercises it, so the signal isn't lost — just deferred to the green-mutation run. If we later want both signals on one run, serialise via a `concurrency` group rather than `needs` + `always()`. Documented as §7 in `docs/runbooks/gitea-actions-gotchas.md`. ## Note on the stuck run Run 582 (the #133 merge) will **not** clear itself and must be force-cancelled from the Actions UI (plain cancel can also stall on this version, gitea#35782). This PR's own run is the first real test of the fix — if `verify-stack` dispatches and runs here, the fix holds. ## Definition of Done - [x] Linked issue (#134). - [x] Conventional Commit referencing the issue. - [ ] CI green — this PR's run is the verification (verify-stack must dispatch). - [x] Runbook updated (gotchas §7). - [ ] Closed by the merging PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #135
This commit was merged in pull request #135.
This commit is contained in:
@@ -144,11 +144,15 @@ jobs:
|
|||||||
# they never co-schedule now the runner has capacity >1. A concurrent Stryker run + full-stack
|
# they never co-schedule now the runner has capacity >1. A concurrent Stryker run + full-stack
|
||||||
# bring-up + Playwright browser on one host is what OOMs the e2e (commit d5e5fa2, #126). The
|
# bring-up + Playwright browser on one host is what OOMs the e2e (commit d5e5fa2, #126). The
|
||||||
# light .NET/frontend jobs have no `needs`, so they still parallelise up to runner capacity.
|
# light .NET/frontend jobs have no `needs`, so they still parallelise up to runner capacity.
|
||||||
# `if: !cancelled()` keeps verify-stack running even when the mutation ratchet fails (so we don't
|
#
|
||||||
# lose its signal) while still honouring run cancellation from the concurrency group above.
|
# No `if: ${{ !cancelled() }}` here (removed in #134): on Gitea 1.27 + act_runner 2.0.0, a job
|
||||||
|
# gated by a status-function `if` (always()/cancelled()) on top of `needs` routes through the new
|
||||||
|
# transitional "Cancelling" state + capability negotiation and never leaves `waiting` — it's never
|
||||||
|
# dispatched (gitea-actions-gotchas.md §7). Default `if: success()` dispatches normally. Cost: a
|
||||||
|
# failing mutation ratchet now skips verify-stack instead of running it anyway; the fix-and-re-push
|
||||||
|
# re-run exercises verify-stack, so we still get the signal.
|
||||||
verify-stack:
|
verify-stack:
|
||||||
needs: [mutation]
|
needs: [mutation]
|
||||||
if: ${{ !cancelled() }}
|
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- uses: https://github.com/actions/checkout@v4
|
- uses: https://github.com/actions/checkout@v4
|
||||||
|
|||||||
@@ -196,3 +196,28 @@ service name; the notif verify harness also registers the sink callback by IP.
|
|||||||
abonnement is registered and refuses it (`no-auth-on-callback-url`) unless it returns
|
abonnement is registered and refuses it (`no-auth-on-callback-url`) unless it returns
|
||||||
**401** without the configured `Authorization`. The verify sink
|
**401** without the configured `Authorization`. The verify sink
|
||||||
(`infra/notification-sink.py`) enforces a bearer token for exactly this reason.
|
(`infra/notification-sink.py`) enforces a bearer token for exactly this reason.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. A job with `if: ${{ !cancelled() }}` (or `always()`) + `needs` sticks in "waiting"
|
||||||
|
|
||||||
|
**Symptom** — after upgrading to **Gitea 1.27** + **act_runner 2.0.0**, one job never
|
||||||
|
starts: the run sits in state `waiting` forever, the job has **no logs** (never
|
||||||
|
dispatched to a runner), and the other jobs finish normally. `main` stays pending/red.
|
||||||
|
Seen on the `verify-stack` job (#134).
|
||||||
|
|
||||||
|
**Why** — Gitea 1.27 reworked cancellation/aggregation: a job gated by a
|
||||||
|
**status-function `if`** (`always()` / `cancelled()` / `!cancelled()`) on top of
|
||||||
|
`needs` now routes through a new transitional **`Cancelling`** job state plus a
|
||||||
|
server↔runner **capability negotiation** ("Requires Gitea Runner 2.0.0"). On the
|
||||||
|
1.27 + 2.0.0 pairing that handshake doesn't resolve for such a job, so it's never
|
||||||
|
offered to a runner and never leaves `waiting`. Jobs with no `if`/`needs` are
|
||||||
|
unaffected. (Related upstream: go-gitea/gitea#31074, #27116, #35782.)
|
||||||
|
|
||||||
|
**Fix** — don't gate a `needs` job with a status-function `if`. Use the default
|
||||||
|
`if: success()` (i.e. omit the `if`). If you need "run even when an upstream job
|
||||||
|
fails", prefer serialising with a `concurrency` group over `needs` + `always()`.
|
||||||
|
|
||||||
|
**Also** — a run already stuck this way will **not** clear itself; force-cancel it
|
||||||
|
from the Actions UI (plain cancel can also stall on this version, #35782). Push the
|
||||||
|
workflow fix to produce a fresh run.
|
||||||
|
|||||||
Reference in New Issue
Block a user