# BGS-1 — moving the digest to the Windows box gated six jobs behind a new one-minute job, and four detectors that lived inside those jobs have no box-side equivalent; #251's new tests assert what was added and are blind, by construction, to every one of the four removals status: open · raised panel run 40 (2026-10-19; change-reviewer catch on #251, merged 2026-10-18 — a defect row, **not** a research or capture proposal, so it does **NOT** discharge the frontier scout's cadence clock; same treatment as LAB-1, BTB-1 and CQA-3) · class: loudness/safety (P0, G2) with a moat leg (G4) and an instrument-calibration leg (P2/G5 — tests that cannot fail) · effort ~1.25 pd for all four legs, each independently shippable · horizon: legs (a) (b) (c) activate **the day an admin installs the box's `digest` task** — latent today, imminent; leg (d) is live now --- ## Plain-language summary for an owner reading one paragraph #251 is good work and its motivation was right: the Windows box does the real work, so GitHub should stop re-doing it. It does that by putting a new one-minute job, `box`, in front of each GitHub leg — the job asks `main` whether the box already committed today's record, and the expensive leg skips if it did. The trouble is **what else lived inside those legs.** Six jobs are now gated, and four of them carried something the box does not replicate: the *only* place 552 of this repo's 1,795 tests ever run; the *only* machine-readable channel that says a sleeve did not trade; the fix for the 2026-10-07 race that emailed "meanrev did not run" while meanrev was mid-flight and went on to submit 62 orders; and — for the gate itself — any alarm at all. None of this is hypothetical bad luck: #251 shipped 18 new tests, and **all four removals pass them**, because the tests check that the gating was added correctly rather than that nothing was lost. Three of the four controls below were executed; the fourth is a GitHub-semantics claim I could not run here and have labelled a hypothesis rather than asserted. --- ## Leg (a) — 552 of 1,795 tests (31%) and the entire `web` suite now run NOWHERE on a normal weekday, and the alert for that is exempted **The defect.** `fleet-digest.yml:243-245` added to the `observability` job: ```yaml observability: needs: box if: ${{ needs.box.outputs.ran != 'true' }} ``` That job is the **sole** executor of the reporting/research test tier. Enumerated over every pytest invocation in the repo: ``` $ grep -rn 'pytest' .github/workflows/*.yml scripts/win/*.py fleet-digest.yml:266 python -m pytest tests/ -q -m "observability or research" <- the ONLY one paper-trading.yml:100 -m "not observability and not research" paper-trading-vrp.yml:117 -m "not observability and not research" paper-trading-meanrev:132 -m "not observability and not research" scripts/win/session.py:423 -m "not observability and not research" ``` (`fleet-digest.yml:301` and `:304` look like a second executor but sit inside `alert-observability`'s `hint:` text — a reproduce-locally instruction, not a run step.) The box runs the **complement**, and only for trading: `session.py:421-424` is `not observability and not research`, and `run_task` (`session.py:555`) calls `test_gate()` only `if task == "trading"` — so **the box's `digest` task runs no tests at all.** Tier size, measured at HEAD `6c8d2d0`: ``` $ python -m pytest tests/ -q -m "observability or research" --collect-only 552/1795 tests collected (1243 deselected) ``` The `web` npm suite (`tests/mcp.test.ts`, surface completeness, the scale-free contract) lives only in that job and has no other home. `alert-observability` (`fleet-digest.yml:299`) is `always() && result != 'success' && result != 'skipped'`, so the removal is **silent**. And the job's own header is now false on a box-ok day — it reads (`fleet-digest.yml:240-241`): > *"As its own job it still reds the run and still pages via > alert-observability below — it just cannot hold the mail hostage."* The same header records why the job exists: on 2026-07-27 a missing matplotlib failed a chart assertion and *"killed the digest email for three trading days, taking the fleet's only same-day did-not-run detector with it."* **Negative controls (executed by the change-reviewer lens; the executor map, tier size and header text re-verified by the chair).** A failing `observability`-marked test was planted in a copy of the tree and both selectors run: ``` === the box's EXACT pre-trade selector (session.py:423-424): 19 deselected in 0.06s <- green, nothing ran === the fleet-digest observability job's selector: FAILED tests/test_planted_rendering_bug.py::test_a_rendering_regression_... 1 failed, 18 passed ``` Second control: **both** the `observability` and `site-export` gates were deleted from `fleet-digest.yml` and #251's whole new suite stayed green (`18 passed`). Nothing asserts those gates in either direction. **Harm path.** From the day the box's `digest` task is installed, rendering, digest, lab and `web` regressions land with CI green and nothing paged. That is the exact class that produced the 10-15 mid-sentence truncation and the 10-16 headline cut (#244, still open) — defects in the one email the owner reads, caught only by this tier. The decision corrupted: the steward and the daily audit both read "all green" as "the reporting layer is tested". **Fix (~0.25 pd) and why it is nearly free.** Drop `needs: box` from `observability`. Its own header says *"No secrets and no data cache: every observability test is fixture-based"* and it is capped at `timeout-minutes: 10` — so it is the **cheap** job and was never what drained the Actions allowance #251 was written to protect. Plus one line in the gated/ parity test pinning that this job is **not** box-gated. --- ## Leg (b) — on a box-ok day, "SLEEVE DID NOT RUN" has no channel any reader outside the Windows PC can see **The defect**, verified link by link at HEAD: 1. `session.py:285-287` — the box turns the digest's NO-RUN exit code into success: `if rc == 2: log("...NO RUN"); return 0`. Its docstring is explicit that this is deliberate: *"Exit 2 = the digest is healthy but a live sleeve has no run record today: that is logged, not a failure of the digest."* That is correct **about the digest** and is where the fleet-level consequence begins. 2. `ctx.failed` therefore stays empty, so `session.py:252` stamps **`Box-Result: ok`**. 3. `_box_ran.yml:55` skips only on `ok` → GitHub's `digest` job is **skipped**. 4. `no-run` (`fleet-digest.yml:321-322`, `needs: digest`, `if: needs.digest.outputs.code == '2'`) is skipped with its dependency. 5. The CLI's other two channels are dead on the box: `src/thales/utils/notifications.py:61-62` returns early unless `GITHUB_ACTIONS == "true"`, so the box cannot write an annotation; and the escalation email is withheld by `notifications.email: weekly_digest_only`. Surviving record: one `log()` line in `C:\thales\home\logs\digest.log` — not committed, not fetchable, and unreadable by the steward, which under the 2026-10-07 delegation is the owner's stand-in for exactly these withheld records. **Negative control (executed end to end by the change-reviewer lens, with `fleet digest` stubbed at rc=2 and everything else real — real `session.py`, real jq filter, real `config/fleet.yaml`):** ``` [session] fleet digest: a live sleeve produced no run record today (NO RUN) box Result: Result(sleeve='digest', failed=[], committed=True, detail='') commit the box left: site: daily public data refresh 2026-10-19 Runner: tbx-win Box-Result: ok _box_ran.yml jq over that commit -> n=1 => ran=true => `digest` SKIPPED => `no-run` SKIPPED email_allowed(alert) = False github_annotation outside Actions prints -> (nothing) ``` `grep -rn 'no-run' tests/` returns only CLI exit-contract tests — **nothing inspects the `no-run` job**, which is why its consumer disappearing was not noticed. **Harm path.** This is the precise failure the `no-run` job was built to end, in its own words (`fleet-digest.yml:306-308`): *"the digest detected the outage, said so in the email, and still exited 0, so `gh run list` showed an all-green fleet."* Two triggers have already occurred this month: a box slot not firing (the 10-16 pattern) and a sleeve's step failing closed (10-13 meanrev). Mon–Thu the owner gets nothing by design, the steward sees green Actions and no annotation, and **if vrp is the silent sleeve an open SPY put spread goes unmanaged with no alarm.** **Fix (~0.5 pd).** On `rc == 2`, append to `ctx.failed` (e.g. `no-run`) so the commit stamps `Box-Result: failed (no-run)` → the gate does not skip → GitHub's digest runs as the backup → the existing `no-run` job reds the run exactly as before. Optionally also call `session.alert()` on rc==2 (that path has its own SMTP and does not consult `email_allowed`). Add a test that an rc==2 digest leaves a commit the gate does **not** skip on. --- ## Leg (c) — the 2026-10-07 false-NO-RUN race fix was never ported to the box, and its test still passes because it reads the YAML of a job that no longer runs **The defect.** `fleet-digest.yml:124-183` is the *"Wait for in-flight sleeve runs, then refresh state"* step — a bounded `gh run list` wait, then `git checkout -- data/` and `git pull --ff-only`. It was built after the 2026-10-07 incident in which the digest emailed "meanrev NO RUN" while meanrev was mid-flight and went on to submit 62 orders. `tests/test_execution/test_ops_alerting.py:378-416` pins four of its properties as "each load-bearing". The box replicates none: ``` box STEPS['digest'] = ['fleet digest', 'site export', 'persist'] YAML digest job steps = [checkout, setup-python, Restore data cache, Install deps, Create config/.env, 'Wait for in-flight sleeve runs, then refresh state', 'Send fleet digest', beacon] ``` `boot.py:226` syncs `main` *before* the task, i.e. at 08:30 PT; there is no post-trade refresh, and the box has no `gh` at all. **Negative control (executed by the change-reviewer lens).** #251's PR body advertises `tests/test_digest_on_box.py:68-78` as *"parity with fleet-digest.yml's commands"*. Its regex extracts only `thales` tokens: ``` parity test compares ONLY these tokens: ['fleet digest', 'export-public'] ``` The in-flight/refresh step contains no `thales` invocation, so it is **outside the parity contract by construction**, and the suite is green with the box missing it entirely. `test_no_run_detector_waits_for_in_flight_sleeve_runs` also stays green, because it only parses `fleet-digest.yml`. **Harm path — conditional, and the condition is a day that already happened.** Box-traded days are protected by construction (the digest waits on the runner lock behind the 07:35 PT sequence). The exposed day is the one #251 itself cites: a sleeve fails on the box (`Box-Result: failed`) → GitHub's backup trades it at 15:30Z + queue delay, i.e. **after** the box's 08:30 PT = 15:30Z digest. The box then renders NO RUN for a sleeve that traded an hour later, swallows it (leg b), commits `ok`, publishes `web/public/data` without that trade, and the GitHub digest that *would* have waited and re-read state is skipped. On a Friday that wrong "NO RUN / 0 trades" is what the owner reads; on thales.report it is what the public page shows. Precedent: 10-13. **Fix (~0.5–1 pd), cheapest first.** Move the box's `digest` slot after the backup legs' observed completion; or port the `git pull --ff-only` refresh half into `STEPS["digest"]` as a pre-step (the wait needs `gh`, the refresh does not, and the refresh is the half that matters); or widen the parity test from "thales tokens" to "every run step accounted for or explicitly waived" — which closes the whole class rather than this instance. --- ## Leg (d) — the gate is unalerted, and at job level it is fail-CLOSED against its own header **The defect.** `_box_ran.yml:16-18` promises *"FAILS OPEN: if the API call fails, `ran` is false and the backup runs as it always did."* True at **step** level — eight synthetic scenarios (gh exit 1, empty stdout, `null`, `true`, garbage, count 0/1, manual dispatch) all resolve to `ran=false` with the `::warning::could not read main's commits` line. But the gated jobs are `needs: box` + `if: ${{ needs.box.outputs.ran != 'true' }}` with **no `always()`** (`paper-trading.yml:28-29`, `-meanrev:49-50`, `-vrp:47-48`, `skew-snapshot.yml:31-32`, `fleet-digest.yml:41-42`), so per documented Actions semantics a job whose `needs` dependency **fails** is skipped. The repo knows the rule: `site-export` and every `alert-*` job in the same file use `always() && …`, and `site-export`'s is the only gate written `always() && needs.box.outputs.ran != 'true'`. Then `skipped` is laundered: it had **two** causes with opposite meanings for the backup's health, and the alert layer is told to ignore both. | cause | means | alerted? | |---|---|---| | gate ran, `ran=true` | the box did today's work — benign | correctly silent | | **gate died** (allowance refusal, runner loss, `timeout-minutes: 3`) | **GitHub was never asked; the backup did not run** | **silent — wrongly** | `skipped` was **unreachable** on these jobs until Sunday. The `!= 'skipped'` clause dates to 2026-08-20 (`git log -S` on `paper-trading.yml` returns exactly #105, the runner-loss backstop), when the `trade` job had no `needs:` and no `if:`: ``` $ git show 6c8d2d0^:.github/workflows/paper-trading.yml | sed -n '/^jobs:/,/runs-on/p' jobs: trade: runs-on: ubuntu-24.04 ``` Eleven alert jobs carry `!= 'skipped'`; six guard a now-gated job. **`_box_ran.yml` has no alert job anywhere** — one unalerted SPOF in front of five workflows. **The capture arm is the uncovered half, and its loss is permanent.** `skew-snapshot.yml`'s gated `snapshot` job contains, in order: *Snapshot option skew (full universe)* · *FINRA short-sale capture* · *Shortability snapshot* · *Persist* · **_Capture integrity QA_** · *beacon*. A gate death skips all six, so three un-backfillable streams are not captured **and the QA that would notice is inside the thing that was skipped.** There is no equity curve to fail to advance, so `heartbeat-monitor.yml` (correctly left ungated by #251 — the sleeves' surviving dead-man) does not cover it; and the four trading/capture healthchecks (`thales-momentum`, `-meanrev`, `-vrp`, `-skew`) were **deleted on healthchecks.io 2026-10-02** by owner decision (`ops/windows-host/README.md`), so there is no out-of-band beacon either. `NORTHSTAR.md` §1 asset 2: *"A missed day is a permanent notch."* **Negative control (executed).** The suite does not merely leave this unpinned — **it pins it open.** `tests/test_execution/test_ops_alerting.py:94`, inside `test_alert_job_condition_actually_fires_on_a_lost_runner`, parametrised over all six alerting workflows: ```python assert f"needs.{work_job}.result != 'skipped'" in flat ``` It is the **only** assertion in that test with no rationale message — every other carries a paragraph naming the incident it encodes. Line 354 pins the same clause for `site-export`. `python -m pytest tests/test_execution/test_ops_alerting.py -q` → `60 passed`. Separately, adding `always() &&` to `paper-trading.yml`'s gate leaves #251's 18 new tests green, while *inverting* the gate is caught — so the test bites on inversion and is blind to the fail-open property the header advertises. #251's own new test comment documents only the benign cause, which is the blind spot in one line (`test_ops_alerting.py:321`): ```python # `box` (2026-10-19): the whole job is skipped when tbx-win already exported today. ``` **What was verified empirically, and the one link that was not.** Refusal produces §4c's platform-refusal signature, including on reusable-workflow caller jobs (which `box` is) — run `39191451973`, 2026-10-19 02:45Z: ``` watch | failure | runner_id=0 | steps=0 alert / email | failure | runner_id=0 | steps=0 ``` annotation verbatim: *"The job was not started because recent account payments have failed or your spending limit needs to be increased."* **NOT verified here:** that GitHub reports a job whose `needs` failed as `skipped` rather than `failure`. No Actions runner is available and `act` is not installed, and no gated workflow has fired since #251 merged (first scheduled firings 2026-10-19 15:30Z, after this run). Labelled a hypothesis; it is this leg's primary kill criterion below. **Fix (~0.25 pd).** Prefix the four gates with `always() &&` (matching `site-export`, which already does), add one `alert-box` job per gated workflow (`needs: box`, `if: ${{ always() && needs.box.result != 'success' }}`) so the record says *"the gate could not run"* rather than *"the sleeve failed"*, give line 94's bare assertion the rationale it never had, and assert `always()` in the gated test. --- ## Pre-registered kill criteria — per leg, so a partial fix closes only its own leg - **(a)** dies when `grep -n 'needs: box' -A2 .github/workflows/fleet-digest.yml` shows the `observability` job ungated **and** a test asserts it stays so; or on an owner ruling that the reporting tier may run weekly rather than daily, in which case the weekly cron is the fix and this leg closes with it. - **(b)** dies when an rc==2 digest on the box produces a record readable off the box — a `Box-Result: failed (no-run)` commit, a committed ledger row, or an email — demonstrated by a test. It does **not** die on "the Friday email will say so": leg (b) is about Mon–Thu, and about vrp. - **(c)** dies when either the box's digest runs after the backup legs, or the refresh half is ported, or the parity test accounts for every run step. Also dies if the owner retires GitHub's trading backup legs outright — then there is no later-trading leg to race. - **(d)** dies **immediately** if a gated workflow firing on or after 2026-10-19 15:30Z with `box` dead reports the gated job as `failure` (or anything but `skipped`): then the alert fires as designed and there is no hole. Checkable from the Actions API by the next panel run. It also dies if CQA-3 is built such that `thales capture-qa` reds a session with no capture at all regardless of which leg was meant to take it — that closes the moat arm, and what remains is a sleeve-level duplicate of the heartbeat. - **The whole row** dies if the Actions backup legs are retired rather than gated. That is a live option once the allowance question is settled, and it would be a cleaner answer than four fixes. ## Adjacent, flagged not claimed - **DSP-1** (dismissed) — its panel-39 append already says *"the `Runner:` trailer needs **a** reader — a session that trades from the backup leg while the box is silent is the signal, and it is currently written down and read by nobody."* #251 **partly satisfies** that and deserves the credit: whether the gated job ran or skipped is now a machine-readable record of which runner traded. But the reader it added **acts** (skips) rather than **reports**, so DSP-1's channel-neutral reopen condition (*"a session trades from the backup leg on any day where no human or agent reader is notified by any channel within one trading day"*) is made **more** likely by leg (b), not less. Whether leg (b) belongs here or as a dated append on DSP-1 is the steward's `git mv` per `ops/QUEUE_TRIAGE.md:100-102`, not the panel's. - **LAB-1** (open) — the sibling defect in the same gate family (marker deselection happening after collection). Cited as precedent for treatment, not duplicated. - **CQA-3** (open) — blind to the capture *instant*; leg (d) is blind to the capture *not happening at all*. Kill criterion (d) names the seam. - **Docs, no row:** `ops/windows-host/README.md:57` (added by #251) says the digest is a tbx-win task, while `:59` two lines later still says *"Fleet digest … stay on GitHub — the observers must keep working when tbx-win is down"* — same table, opposite answers, and `:59`'s rationale is precisely what legs (a)–(c) are about. `CLAUDE.md:388` still describes `fleet-digest.yml` as running the *"public-site refresh, no-run detector, observability suite"*, none of which it does on a box-ok day. Both are green against `test_docs_consistency`. ~0.1 pd, and the README row is what an interactive session or the steward reads when the box misbehaves. - **Minor, no row:** `session.py:234`'s new docstring cites `scripts/box_ran_today.py`, which does not exist; the real gate is `_box_ran.yml`, cited correctly 18 lines later. And no detector pins the runner image (`grep -rn ubuntu tests/` → nothing), so the next workflow a routine adds will default to `ubuntu-latest` and silently land on Ubuntu 26 — one line in `test_ops_alerting.py` closes it. ## Reviewed in the same pass and found CLEAN The push-fix half of #251 (BOT identity on every git call, `pull_rebase()` abort) — each half reverted in a copy and all three of its tests bite, as the commit message claims; correctly `observability`-marked so it can never fail-close the box's own gate. `_box_ran.yml`'s jq filter — real jq over six commit shapes; `Box-Result: failed (trade)`, a `Runner: github-actions` commit, another sleeve's subject and a `Revert "…"` all correctly rejected. The UTC date agreement between the box's subjects and the gate's `date -u +%F`, holidays included. `detect_runner_loss.py` (keys strictly on `cancelled`; skipped jobs add no false positive). The `_last_beacon_date_git` fallback, which is necessary — without it every routine reads `unknown` and #249's steward-dark alarm could never fire. #249's detector keys on the literal `beacon/steward` string in `ops/ROUTINES.md:44`, so it cannot be disarmed by renaming the routine, as its comment claims. And **#248 ON `main`**: 15 files / 20 jobs, `grep -rn ubuntu-latest .github/` → none, no composite actions, no matrix strategies, every reusable-workflow caller carrying its own pinned `runs-on`; the commit message's "14 workflows (19 jobs)" is exact at its parent. **CORRECTION, appended 2026-10-19 14:50Z — "#248 in full" was wrong, and this run's own beacon push is the proof.** This row first recorded #248 as complete. It is complete **on `main`**, which is what both of this run's lenses greped — and that is the wrong ref for one workflow. `routine-beacon.yml` fires on `push: branches: ['beacon/**']`, and GitHub resolves a push-triggered workflow **from the pushed ref**. All seven `beacon/*` branches carry a full stale repo tree, and their copy still reads `ubuntu-latest`: ``` main: .github/workflows/routine-beacon.yml:48 runs-on: ubuntu-24.04 beacon/panel: .github/workflows/routine-beacon.yml:48 runs-on: ubuntu-latest ``` Measured on run `39263175000` — **panel run 40's own 13:47Z beacon push**, twelve hours after #248 merged: ``` relay | failure | labels=["ubuntu-latest"] | runner_id=0 ``` Found by the **weekly audit (#253, finding ❌3, opened 14:27Z today)**, which reproduced it independently; the chair verified it above rather than taking it on report. Credit to that audit, and the miss is this panel's: two lenses greped `main` and neither asked which ref executes. **Honest severity, and why it is not a leg of this row.** The relay is a one-step curl and will most likely survive Ubuntu 26, so the image is the small half. The durable defect is that **no future change to `routine-beacon.yml` on `main` can ever reach the push leg** — and the push leg is the only one a cloud routine can use (`repository_dispatch` is denied to routine sessions, incident 2026-08-10). The liveness relay is frozen at whatever seven stale branches happen to hold. That is a different object from #251's gating and belongs to #253, which recommends recreating the branches from `main` or making them orphan branches holding only `last-run.txt`, plus a test asserting each beacon branch's copy matches `main`'s — and correctly declines to do it, since rewriting seven liveness branches is an outward-facing act on the only signal that proves a routine ran. ## The slot this costs `open/` held 11 of 12 when this was raised; this makes it **12, at the cap**. The four legs were deliberately filed as ONE row rather than four: they share a root cause (the move gated jobs whose detectors have no box-side twin) and a single blind spot (#251's tests assert what was added), and they would be built together. The run's other two candidates were likewise kept out of slots — the capture-instant finding went in as an annotation to RAD-1 (same single owner ruling) and SCR-1 was dismissed with live reopen conditions (`dismissed/scr-1-fired-scout-criterion-unrecorded.md`). --- ## Activation note, appended 2026-10-19 15:40Z — this row's own trigger has FIRED The header above said legs (a) (b) (c) *"activate the day an admin installs the box's `digest` task — latent today, imminent."* **That day is today.** The deployment landed and the box ran all four of its tasks on schedule, with real git trailers (not a grep over quoted commit text): ``` 2026-10-19T15:30:23Z trailers=[tbx-win, ok] site: daily public data refresh 2026-10-19 2026-10-19T14:58:15Z trailers=[tbx-win, ok] state: meanrev paper run 2026-10-19 2026-10-19T14:56:50Z trailers=[tbx-win, ok] state: paper trading run 2026-10-19 2026-10-19T14:37:09Z trailers=[tbx-win, ok] state: vrp paper run 2026-10-19 ``` 15:30:23Z is 08:30 PT — the new `digest` task firing exactly on its registered slot. So `install-tasks.ps1` was re-run, `boot.py` accepts `digest`, and `session.py` is stamping `Box-Result`. **Good news first: the Friday 2026-10-23 digest will send**, which was the open question this run pushed to the owner. **And that is precisely what arms legs (a)–(c).** From today, a `Box-Result: ok` record exists every weekday, so `_box_ran.yml` resolves `ran=true` and GitHub's `digest`, `observability` and `site-export` jobs skip by design. Leg (a) is therefore live in the strongest form available: **today the reporting/research tier ran nowhere at all** — the box's digest task runs no tests, and GitHub's `observability` job could not run even if ungated, because the account-level refusal is still in force (every run through 15:30:28Z is `runner_id=0`, including `Main Push Watch` on the box's own digest commit). The two causes are independent and both must be fixed: the refusal lifts by itself (~2026-11-01), after which the **gate** keeps that tier skipped every day until leg (a) is addressed. Nothing about today's successful deployment closes any leg of this row; it starts their clock. One honest consequence for leg (d)'s kill criterion: it asks whether a gated job whose `box` dependency *failed* reads `skipped` or `failure`. Today could not answer it — the box succeeded, so the gate's fail path was never taken on a live firing. The criterion stands unresolved and still cheap, and the next panel run should read it off the first firing where the box record is absent. ## ANNOTATION 2026-10-20 (panel run 41) — LEG (d)'s KILL CRITERION IS ANSWERED: `skipped`, on all five workflows, with the alert skipped too. The criterion was answerable from 2026-10-19's own firings and run 40 read the wrong signal. Plus a new leg (e): the box's own alarm is withheld on the written premise that THIS leg covers it Annotation only; the body above is untouched. ### Leg (d) — CONFIRMED, not hypothesis. Seven gated jobs, five workflows, one evening Run 40 closed with: *"it asks whether a gated job whose `box` dependency *failed* reads `skipped` or `failure`. Today could not answer it — the box succeeded, so the gate's fail path was never taken on a live firing."* **That is wrong, and the way it is wrong is the finding.** The box succeeding sets `ran`; it says nothing about whether the `box / check` *job* fails. On 2026-10-19 the gate job failed on **every one of the five workflows that call `_box_ran.yml`** — the account-level refusal takes the runner before any step runs — so the fail path was taken five times that evening. Read off the Actions jobs API (`conclusion`, not inference): | run | workflow | `box / check` | gated job | its `alert` | |---|---|---|---|---| | 39321205783 (20:40:02Z) | Paper Trading | **failure** | `trade` **skipped** | **skipped** | | 39321409442 (20:41:27Z) | Paper Trading (meanrev) | **failure** | `trade` **skipped** | **skipped** | | 39321400775 (20:41:23Z) | Paper Trading (vrp) | **failure** | `trade` **skipped** | **skipped** | | 39337754309 (22:50:52Z) | Skew Snapshot | **failure** | `snapshot` **skipped** | **skipped** | | 39321763264 (20:43:57Z) | Fleet Digest | **failure** | `digest`, `observability`, `no-run` **skipped** | `alert`, `alert-observability` **skipped** | So: **`skipped`.** And because every `alert:` job tests `needs..result != 'skipped'` (`paper-trading.yml:447`, `-meanrev.yml:259`, `-vrp.yml:208`, `skew-snapshot.yml:151`, `fleet-digest.yml:290,413`), the alarm skips with it. Leg (d) is live and fleet-wide, not digest-local: the three sleeves' backup trading legs and the skew capture are inside it. ### The negative control ran itself, in the same run, in the same minute `fleet-digest.yml:337-339` is the control, and it is the only gated job in the repo that carries one: ```yaml site-export: needs: [box, digest] if: ${{ always() && needs.box.outputs.ran != 'true' }} ``` In run 39321763264, with the identical failed `box / check` upstream, `site-export` → **failure** (it ran, then hit the refusal) and `alert-site-export / email` → **failure** (it fired). The six gated jobs without `always()` → `skipped`, silent. Same commit, same gate, same minute: the single token `always() &&` is the whole difference between a loud day and a silent one. No code change was needed to run this control. ### The mechanism is documented in this repo, 150 lines below the job that omits it `paper-trading-vrp.yml:200-205`, on the `alert` job: > ``always() &&`` **is MANDATORY**: a job `if:` with no status-check function > carries an implicit "all needs succeeded" and would never be reached. `paper-trading-vrp.yml:48-50`, the `trade` job in the same file: ```yaml trade: needs: box if: ${{ needs.box.outputs.ran != 'true' }} ``` The knowledge was in the file. The gate was written without it. Identically at `paper-trading.yml:27-29`, `-meanrev.yml:50-52`, `skew-snapshot.yml:32-34`, `fleet-digest.yml:40-42` and `:244-246`. ### Fail-CLOSED at job level, against the gate's own capitalised header `_box_ran.yml`'s header promises: *"**FAILS OPEN:** if the API call fails, `ran` is false and the backup runs as it always did."* The *step* honours that — `|| n=""`, a `::warning::`, `ran=false`. The *job* does not: when the job dies before writing its output (refusal, runner loss, the 3-minute `timeout-minutes`, a `set -uo pipefail` abort), every consumer's implicit "all needs succeeded" skips the leg. The header describes the step's contract and is read as the system's. ### Why run 40 misread it — the ambiguity has already claimed a reader Run 40 flagged that `skipped` "now has two opposite causes." It does, and the two are **indistinguishable in the job list**: box-ran-fine → `skipped` by design; gate-broke → `skipped` by failure. Run 40 checked the box's commit trailers (which showed the box fine) and concluded the fail path was untaken, while five gate jobs were failing that same evening. The first thing this ambiguity cost was the panel's own reading of its own kill criterion. That is the strongest available argument that the two causes must not share a word. ### Leg (e) — NEW. The box's own alarm is withheld on the written premise that this leg covers it, and both halves are dark right now `scripts/win/boot.py:179-184`, docstring of `send_alert`, and the code under it: > *"Withheld under the weekly-digest-only policy: **the GitHub backup leg > covers a box that cannot start**."* ```python if email_policy() != "all": log(f"alert withheld from email (owner policy): {subject}") return ``` `ops/windows-host/README.md:41` names the same compensating control: *"If an update ever reboots the box during a run, GitHub's backup leg covers the session."* The compensating control is the leg this row shows is skipped-and-silent whenever `box / check` fails. Under the refusal that is **every firing, every workflow, every day**. So for the nine trading days 2026-10-20 → 2026-10-30 (the refusal lifts ~2026-11-01; the September refusal ran 09-21 → 09-30, consistent with a 1st-of-month reset), if the box misses a slot: the box does not mail, because the backup covers it; and the backup does not run and does not alert, because its gate failed. **Neither automated alarm speaks** (see the correction below for the two after-close readers that do). The daily audit of 2026-10-19 raised the withholding as `NEEDS-OWNER` and described the backup as "refused" — understating it: a refused leg that reported `failure` would have alerted. It reports `skipped`, which the alert layer exempts by design. Not hypothetical: the steward's 2026-10-16 record has the box missing its second slot that week (skew, past the B4 grace), later traced to a push with no committer identity (`win/rebase-identity-fix`, folded into #251). A box that misses a slot inside this window is a demonstrated failure mode, not a tail. **CORRECTION TO THIS ANNOTATION'S OWN FIRST DRAFT, and it matters.** I drafted this as "neither channel speaks" and as a new `NEEDS-OWNER` line for Friday. Both were wrong, and the steward's 2026-10-19 record (`research/steward/2026-10-19.md:104`) is why: > *"NEEDS-OWNER: decide whether the box's failure emails stay switched off > while GitHub's backup leg is down — the code justifies the silence by saying > 'the GitHub backup leg covers it', and until ~11-01 that leg cannot run. — My > recommendation: leave `notifications.email: weekly_digest_only` as it is … > A box failure is still found the same evening: the daily audit checks at 3pm > PT and I check at 4pm PT, both reading the box's commits. The cost is that a > failure is found after the close instead of during the session … The one > exposure is vrp's open SPY put spread (expires 11-20), which no one would > manage that day."* So: **the ask already exists, dated before mine, with a recommendation and a priced residual — and it names a third channel I had missed.** The daily audit (~22:00Z) and the steward (~23:00Z) are claude.ai routines that read the box's commits out of git; they depend on neither the gate nor healthchecks, which is why they are still working while everything on GitHub is refused — this panel run is itself the evidence. Detection therefore does not vanish, it **moves from during-session to after-close**, exactly as the steward priced it. Two consequences, and I am taking both: - **No new owner ask from this run.** The steward is explicitly rationing (`:119`: *"NEEDS-OWNER today because there are four asks already"*). A fifth line restating its second would spend the one channel NOL-1 just finished repairing. The correct move is to leave ask #2 standing and attach this row's evidence to it. - **This annotation's live-exposure claim is narrowed accordingly.** What is dark for the nine trading days 2026-10-20 → 2026-10-30 is the *automated, same-session* alarm path — the box's email (withheld) and the backup leg's `alert` job (skipped). What survives is two after-close readers. The blackout-window half of leg (e) is therefore **already owned and answered by the steward**; what this row adds is below, and it is the half that outlives 2026-11-01. ### Leg (e), the part that is NOT already owned — the hole survives the blackout Everything the steward and the 10-19 daily audit wrote about this is scoped to the refusal window ("until ~11-01", "~9 trading days"). The gate defect is not. After the allowance resets, `box / check` succeeds and the leg works — **until the next time that job fails for any reason**, at which point the same silent skip recurs for that day, with no blackout to explain it and nobody looking. Causes that need no billing outage: a transient GitHub API error on the one `gh api` call, runner loss, the 3-minute `timeout-minutes`, a `set -uo pipefail` abort before the output is written. Precedent for exactly this class of one-API-call job crashing transiently is open in this repo right now: **PR #245 — the main-push watcher crashed on a transient API error.** That is why the fix below is worth its 0.15 pd even though the loud half of the problem expires by itself in nine trading days. ### Leg (e), second part — the documented cover for the DELETED skew healthcheck is two mechanisms that both sit inside gated jobs, leaving one reader This is the part that compounds, and no single record states it. Trace what watches a missed skew capture — the asset NORTHSTAR §1 calls irreplaceable, where a missed day is a permanent notch: 1. **Its own external alarm is gone.** `RUNBOOK.md:110`: *"~~`HEALTHCHECK_URL_SKEW`~~ — **CHECK DELETED 2026-10-02** … a missed capture day is still unrecoverable at any price, but nothing external watches for one; cover is the fleet digest's no-run detector + `thales capture-qa`."* The owner's call, recorded; the question is only whether the named cover exists. 2. **`thales capture-qa` is inside the gated job.** `skew-snapshot.yml:117-122` runs it as a step of `snapshot`, and `snapshot` is gated at `:32-34` with no `always()`. Confirmed skipped in run 39337754309. 3. **The no-run detector is inside the gated job too**, and fails the other way on a good day: `fleet-digest.yml:313-315`'s `no-run` needs `digest` (confirmed skipped), and on a box-ok day the box's own digest runs, where leg (b) shows `session.py:285` turns the NO-RUN exit 2 into `Box-Result: ok`. 4. **healthchecks.io as a layer sends nothing.** `ops/ROUTINES.md:215-217`: *"healthchecks.io silent since 2026-10-07 — no notification channel, so a missed ping alarms nowhere; the fleet digest's DARK detection and the steward are the live readers."* `RUNBOOK.md:82` is blunter: *"healthchecks.io sends NOTHING."* So of the two readers ROUTINES.md names, the first cannot speak in either branch — gate fails → skipped; box fine → `Box-Result: ok`. **The steward is the only surviving channel**, and it reads run *annotations* that `_alert.yml` writes (`_alert.yml:88-94`) from alert jobs that are themselves `skipped`. It caught the 10-16 skew miss by reading records, which is the system working — but by inspection, not by alarm, and it was never designed as the sole channel. **This is not a new row and not a new idea; it is the blast radius of leg (d) plus leg (b).** It is recorded here because the fix priority follows from it: `always() &&` on `skew-snapshot.yml:34` is the cheapest single token in this repo that restores an alarm on a permanent-loss asset. **Kill criterion:** falsified if any channel outside the gated jobs can be shown to report a missed capture day — name it and this part dies. The check is a grep for a capture-miss reader outside `skew-snapshot.yml`'s `snapshot` job and `fleet-digest.yml`'s `digest`/`no-run` jobs; I found none. ### The instrument-calibration part: the suite asserts this exact rule in two places and in none of the seven that now break it The sharpest version of leg (d). `tests/test_execution/test_ops_alerting.py` already encodes the rule, in a comment that could be the fix's own commit message (`:76-81`): ```python # A job-level `if:` with NO status-check function carries an implicit # "all needs succeeded", so the comparison would never be evaluated. assert "always()" in flat, ( f"{wf}: `if:` must contain always(). Without it the alert job inherits " f"an implicit success requirement and can never run after a failure." ) ``` It asserts `always()` on **every `alert` job** (`:71-85`, parametrized over `ALERTING_WORKFLOWS`) and on **`site-export`** (`:324`). It asserts it on **none of the seven work jobs the new gate feeds**. And `test_site_export_is_its_own_alerted_job_not_a_swallowed_step` was *edited by #251 itself* — `:321` carries the new comment *"# `box` (2026-10-19): the whole job is skipped when tbx-win already exported today."* and `:322` was widened to `assert se.get("needs") in ("digest", ["box", "digest"])`. So the author of #251 opened the one test that asserts this rule, updated it for the gate, preserved `always()` on the job that had it — and added six sibling gated jobs without it, in the same PR, passing all 18 new tests. This is BGS-1's thesis in one file: **the tests assert that the gating was added correctly, and the rule they already encode was never pointed at the jobs the gating was added to.** **Which makes the repair cheap and symmetric, and it is the fix I would put to the owner first.** `ALERTING_WORKFLOWS` (`:33-40`) is already the exact (workflow → gated job) map: ```python ALERTING_WORKFLOWS = { "paper-trading.yml": "trade", "paper-trading-meanrev.yml": "trade", "paper-trading-vrp.yml": "trade", "fleet-digest.yml": "digest", "skew-snapshot.yml": "snapshot", "heartbeat-monitor.yml": "heartbeat", # not box-gated } ``` One new parametrized lint over that map — skip the entry whose job is not box-gated, assert `always()` in the rest — plus an explicit case for `fleet-digest.yml`'s `observability` (the one gated job absent from the map). Then the one-token yml change in seven places. **~0.15 pd total, and the lint is what makes it stay fixed**; the yml change alone would regress on the next workflow somebody gates. **Negative control for the lint (pre-registered):** strip `always() &&` from `fleet-digest.yml:339`'s `site-export` — the one job that currently has it — and the new lint must go red. If it stays green it is testing the wrong jobs, which is the defect this annotation is about, reproduced in its own fix. ### Both controls EXECUTED this run (the earlier draft only proposed them) **Control 1 — the defect is invisible to the suite that owns the rule.** With all seven gated jobs exactly as merged: ``` $ python3 -m pytest tests/test_execution/test_ops_alerting.py -q 60 passed in 18.75s ``` **Control 2 — the test is not broken; it is aimed at 2 of 9 jobs.** Strip `always() &&` from the one gated job that has it (`fleet-digest.yml:339`, `site-export`), change nothing else, re-run the same file: ``` FAILED tests/test_execution/test_ops_alerting.py:: test_site_export_is_its_own_alerted_job_not_a_swallowed_step tests/test_execution/test_ops_alerting.py:323: AssertionError 1 failed, 59 passed in 16.13s ``` Same omission, same file, same `pytest` invocation: **red on the one job the lint covers, green on the six it does not.** The working tree was restored immediately (`git status --porcelain .github/workflows/` clean); nothing was committed. That pair is the whole row in two commands. The rule is encoded, the assertion is live and does fail when violated, and it is pointed at two of the nine jobs that need it — so #251 could add six violations and ship green. The repair is to widen the lint's aim, not to write a new one. ### One dated consequence for another open row, found by this run's queue lens NOL-1's pre-registered kill criterion fires on **Friday 2026-10-23** and needs a digest to actually send to fire at all. Since #251 the Friday send's primary is the box (`ops/windows-host/README.md:57`, task `\thales\digest`, Mon–Fri 08:30 PT), and the box does mail it — `scripts/win/session.py:283` runs `thales fleet digest --fail-on-no-run --email`. GitHub's `digest` job is the backup, and per leg (d) it is `skipped` whenever `box / check` fails, which is every firing under the refusal. So **Friday's digest has exactly one carrier and no working backup.** If the box misses Friday, NOL-1's criterion neither fires nor lapses — it *stalls*, and that row's expiry silently rolls a week. Recorded here rather than in NOL-1 because the cause is this row's gate, not that row's renderer. --- ## ANNOTATION 2026-10-20, 16:00Z (panel run 41, self check-in; its "live instance" WITHDRAWN 2026-10-21 by panel run 42 — see the first subsection) — a THIRD reason the compensating control does not exist: the GitHub backup leg is, in practice, a POST-CLOSE leg Annotation only; the body above is untouched. Written from a self check-in a few hours after this run's PR was opened. ### The "live instance" — WITHDRAWN IN FULL, 2026-10-21 (panel run 42) **This subsection claimed "No sleeve traded on 2026-10-20." That was false.** All three sleeves traded on time, on the box: vrp `3ac4abe` 14:36:52Z, momentum `4605c2d` 14:56:31Z, meanrev `7d6f368` 14:57:46Z — every one carrying `Runner: tbx-win` and `Box-Result: ok`, pushed together at 14:57:55Z, a full hour before the 15:58Z reading that called them 57–81 minutes overdue. All three equity ledgers end `2026-10-20` (13652.81 / 10787.4 / 9960.87). Each commit landed **inside** the very window this annotation measured for it, so its own baseline table acquitted the box and was misread. The cause was a stale clone. Run 41 started on its 13:00Z cron and ran `git log` at ~15:58Z against a working copy it had never re-fetched — a ~13:00Z view of the repository, predating all five of that day's commits. The daily audit caught it the same night (#259) and the steward confirmed it independently on a freshly fetched `main`. Verified a third time here before withdrawing, on a fetched clone at `9e28a45`. **Nothing in legs (d) or (e) rests on this.** Both stand on the 2026-10-19 firings, which are independently verified; the withdrawn reading added no evidence to either. The post-close-latency finding below is also unaffected — it rests on measured `created_at` values from the Actions API, not on any clone. Removed rather than kept-with-a-caveat because a false event in a row's permanent evidence base is worth less than nothing: it would be cited later by someone who did not read this paragraph. ### The new finding: the backup leg lands after the close on most days I went to check whether today's backup legs ran or skipped — leg (d)'s second confirmation — and **could not answer it**, which is itself the finding. `CLAUDE.md` warns never to judge a cron late without reading the actual history first, and it is right. `paper-trading.yml`'s configured cron is `30 15 * * 1-5`. Measured `created_at` for its last eight *scheduled* runs: | date | fired | delay vs 15:30Z | vs 20:00Z close | |---|---|---|---| | 10-19 | 20:40:02Z | +5h10m | **after close** | | 10-16 | 18:53:45Z | +3h24m | in session | | 10-15 | 19:18:01Z | +3h48m | in session | | 10-14 | 19:33:23Z | +4h03m | in session | | 10-13 | 20:46:03Z | +5h16m | **after close** | | 10-12 | 22:32:18Z | +7h02m | **after close** | | 10-09 | 20:12:27Z | +4h42m | **after close** | | 10-08 | 20:45:11Z | +5h15m | **after close** | Shared-runner queue delay of **+3h24m to +7h02m**, and **5 of 8 runs were created after the 20:00Z market close.** So at 15:58Z today's backup leg is not missing — it is not due yet, and would be expected somewhere between ~18:55Z and ~22:30Z. Leg (d)'s second confirmation is therefore **pending**, readable from the Actions API this evening by whoever looks next; nothing about it is decided yet, and I am recording that rather than guessing. **But the latency is the point.** `scripts/win/boot.py:181` withholds the box's alarm because *"the GitHub backup leg covers a box that cannot start"*, and `ops/windows-host/README.md:41` says the same of a mid-run reboot. On the measured distribution, that leg does not start until hours after the session it is meant to cover, and on the majority of recent days not until after the close. `paper-trading.yml:226` then runs `thales run --skip-market-check`, so when it does fire late it trades anyway rather than declining. So the named compensating control fails for **three independent reasons**, only one of which this row previously had: 1. the `always()` gate defect — a failed `box / check` skips the leg silently (leg (d), confirmed 2026-10-19); 2. the account-level Actions refusal — nothing runs at all until ~2026-11-01 (leg (e), first half, owned by the steward's ask #2); 3. **queue latency — even fully working and fully funded, the leg is a post-close leg on most days** (new, today). Fixing (1) and waiting out (2) still leaves (3). A same-session backup for a box failure is not what this leg is, and the two places that promise it should stop promising it. ### Deliberately NOT proposed I am not proposing a fix for (3), and not opening a row for it. Moving the backup cron earlier collides with #201's ordering invariant (the crons sit *later* so they can never race the box) and with the 2026-11-02 DST inversion that LAB-1 leg 3 is about — three constraints on one set of four cron lines is an owner-level design question, not a panel edit. What this annotation asks for is narrower and free: **the two sentences claiming the backup "covers" a box failure should say what it actually does**, which is to re-run the day late, after the close more often than not. **Kill criterion for this annotation.** It dies if the measured delay distribution shifts back inside the session — re-measure `created_at` minus the configured cron over the next eight scheduled runs; if the median lands before 20:00Z and no run lands after it, (3) is not real and only (1) and (2) stand. It also dies if someone shows `thales run --skip-market-check` declines to trade post-close by another path, since then the late leg is inert rather than late. Re-measuring costs one API call. ### Correction to this run's own 15:02Z owner notification — ALSO WITHDRAWN, 2026-10-21 That notification told the owner that no sleeve had traded. **There was no miss, so it was false in its premise and is withdrawn in full** — including the paragraph that stood here defending its conclusion as "unchanged and correct." There was nothing for an automated channel to report; detection did not "fall to" the 22:00Z audit; what that read found was a clean 5-of-5 box day and a false alarm in the record. The subsidiary error the paragraph did own stays true but no longer matters: the backup legs had not fired by 15:02Z, and the measured distribution above puts them hours later. The steward's 2026-10-20 record carries "ignore the false 15:02Z alarm" to the owner. One sentence on the habit, because the withdrawal is cheap and the habit is not: **a long-running routine session that reasons about "what the system did today" from its start-of-session clone will manufacture a false outage, and the longer the session runs the more confident the artefact looks.** This run notified the owner of a trading outage that had not happened, on the strength of a `git log` it never re-fetched, and then reported a compensating control as failing in the field when it had not been tested at all. The lesson is now `ops/DAILY_AUDIT.md` §0c (#259); the panel's standing rule from it is that a liveness claim is not publishable without an explicit `git fetch` immediately before the read.