The Actions-refusal signature now matches a healthy day
2026-11-12, daily audit. Observability only — no trading path touched.
What happened
GitHub Actions has refused every job in this repository since some time between 2026-11-06T23:54:45Z (the last green run) and 2026-11-09T14:44:39Z (the first red one). GitHub says why itself, in one sentence, when asked for the failing check run's annotations:
"The job was not started because recent account payments have failed or your spending limit needs to be increased."
That is the third instance of the class: 2026-09-21→09-30, 2026-10-19→10-31
(cleared 11-01 05:47Z), and now. ops/STEWARD.md carries the owner's standing
ruling — never pay for Actions — so the outage itself is status, not a
question. This note is about the two things that are new.
1. The procedure's confirmation step had no negative control
ops/DAILY_AUDIT.md §4c gave three job-death signatures and then said:
get_workflow_run_usage→billable.UBUNTU.total_ms: 0confirms the third.
It confirms nothing. Measured on four runs that were green on 2026-11-06:
| run | workflow | conclusion | run_duration_ms | billable.UBUNTU.total_ms |
|---|---|---|---|---|
| 42314932868 | Heartbeat Monitor | success | 52,000 | 0 |
| 42285910694 | Main Push Watch | success | 26,000 | 0 |
| 42178101006 | Routine Outbox | success | 21,000 | 0 |
| 42299466901 | Fleet Digest | success | 7,000 | 0 |
Every per-job duration_ms is 0 too. The field reads 0 for every run in this
repository whatever happened to it, so using it as a confirmation would
"confirm" a platform refusal on a perfectly healthy day. A test that cannot
fail is not a test — the same defect this shop has found three times in its
own detectors.
2. Box-gating collapsed the duration discriminator as well
The row for platform refusal reads 3–11 s. That band was measured in
September. Since the box-gating change (#185/#186), tbx-win commits the day's
state first and _box_ran.yml gates every real job out of GitHub's backup
legs — so on a normal day the whole run legitimately finishes in seconds.
Green Fleet Digest run 89 (42299466901, 2026-11-06) took 7 seconds:
inside the refusal band. Seven of its eight jobs were skipped with
runner_id: null, runner_name: null, zero steps and no logs — the same four
columns the table assigns to a refusal.
The comparison, on two runs of the same workflow five days apart:
| green, fully box-gated (run 89) | platform refusal (run 92) | |
|---|---|---|
run_duration_ms | 7,000 | 10,000 |
billable…total_ms | 0 | 0 |
| skipped jobs | runner null, 0 steps | runner null, 0 steps |
| non-skipped jobs | box / check: real runner_id, 3 steps | all three: runner_id: 0, runner_name: "", 0 steps |
| job logs | present for the job that ran | 404 for all |
| annotation | none | names the cause |
The corrected test: list the run's jobs and ask whether any job that is
not skipped holds a runner and at least one step. If one does, the
substrate is fine and the failure belongs to the code. If none does, read the
annotation. The annotation is the only instrument in §4c's list that has not
degraded — and §4c already says to read it first, so the section's lead
instruction was right while its confirmation table had gone stale underneath
it.
This is the shape worth remembering: a detector did not break, the system around it changed until the detector's evidence stopped being evidence. Nothing edited §4c; #185/#186 made a healthy day look like the thing §4c was written to recognise.
3. Thales is not what burns the allowance
The obvious remedy — trim Thales' remaining GitHub legs — buys almost nothing.
Billed-minute estimate for this repository over the six green weekdays
2026-11-01→11-06, computed per job as max(1, ceil(duration/60 s)) and zero
for skipped or refused jobs:
| minutes | jobs | workflow |
|---|---|---|
| 32 | 15 | Paper Trading (meanrev) |
| 27 | 50 | Main Push Watch |
| 27 | 15 | Paper Trading (momentum) |
| 23 | 40 | Fleet Digest |
| 17 | 18 | Auto-merge records PRs |
| 15 | 30 | Heartbeat Monitor |
| 14 | 15 | Skew Snapshot |
| 8 | 15 | Paper Trading (vrp) |
| 7 | 7 | Routine Beacon |
| 6 | 6 | Routine Outbox |
| 1 | 18 | Auto-merge weekly audit PRs |
| 177 | 229 | total |
That extrapolates to roughly 650 minutes a month against the ~600 the
tbx-win cutover budgeted for "the backup legs and the small outside observers"
(ops/windows-host/README.md §Why). Thales is on budget. The onset moving
from week 3 (October) to week 1 (November) is therefore account-level, driven
by the owner's other private repositories, which is outside Thales and already
ruled on.
4. What this means for the architecture, stated plainly
The cutover's invariant was "the observers must keep working when tbx-win is
down". For roughly three weeks in four that invariant is now inverted: the
box is up and the observers are dark. Dark during a refusal: auto-merge, the
beacon relay, the outbox mailer, Main Push Watch, the observability test tier,
and the four backup schedule: legs. Unaffected, because they run on the box:
trading, the skew/FINRA/shortability captures, the public-site export and the
Friday digest.
The residual, which no detector covers: with no backup executor and no in-band alarm, a tbx-win outage during a refusal is detected only by a Friday digest that fails to arrive — up to seven days — or by one of the weekly cloud routines. Recorded here rather than escalated, because the standing ruling answers the only question it raises; if the detection latency is itself worth spending on, that is a queue row, not an alert.