2026-08-31 — The displaced-cron regime: three days of trading after the close
Daily audit finding (window 2026-08-27 22:25Z → 2026-08-31 22:11Z). Facts from the GitHub Actions API, git history, committed ledgers, and Gmail.
What happened
The 08-27 scheduled-events outage (zero runs created all day; see the 08-27
audit report on outbox/daily-audit) did not resolve into normal service.
It resolved into a delay regime: every scheduled run since has been
created hours late, while push-triggered runs start within seconds.
| Trading day | Cron (configured) | Run created | Delay | Market state at run |
|---|---|---|---|---|
| Thu 08-27 | momentum 14:35Z | — (then 00:05Z +1d) | ~9.5h | closed, past midnight UTC |
| Thu 08-27 | meanrev 14:40Z | 23:55Z | 9.3h | closed |
| Fri 08-28 | momentum 14:35Z | 23:38Z | 9.1h | closed |
| Fri 08-28 | meanrev 14:40Z | 23:38Z | 9.0h | closed |
| Mon 08-31 | momentum 14:35Z | 20:17Z | 5.7h | closed (17 min past) |
| Mon 08-31 | meanrev 14:40Z | 20:20Z | 5.7h | closed |
| Mon 08-31 | digest 15:50Z | 21:07Z | 5.3h | — |
Push-triggered runs in the same window (auto-merge, outbox relay, beacons) all started instantly — so this is delayed scheduled-event delivery, not runner scarcity or a billing block. The documented "up to ~2h" queue variance assumption in CLAUDE.md is now off by 3–7 hours.
The fleet has not executed during market hours since Wednesday 08-26.
The crons run thales run --skip-market-check by design, so late runs still
submit; day orders submitted after close queue for the next open. Effects,
per the ledgers:
- meanrev is trading T+1 at the open with ballooning churn. Thursday's displaced run (00:03Z) logged 112 rows; Friday's (23:51Z) logged 445 — 222 market-replaces chasing Thursday's unfilled orders plus 222 fresh selection orders, all dated 08-28, all queued to Monday's open; Monday's (20:30Z) logged 504 — 252 replaces + 251 selection + 1 emergency exit, queued to Tuesday's open. Typical daily counts before this were 43–99. Each batch prices a selection made hours after the close it used and fills at the next open. TCA on this stretch will be the measurement of what the delay regime costs.
- Digest truth diverged. The "8/28" digest ran at 00:38Z Friday — before Friday's session — and alerted "meanrev NO RUN" for a day that had not happened (the backwards canary; STAMP-2, panel run 14, #119). The "8/29" digest ran 00:10Z Saturday and reported Saturday: all quiet, 0 trades. Friday's actual 444-order meanrev batch appears in no digest. Monday's digest (21:08Z) is correct again ($34,686.14 vs ledger sum $34,686.27 ✓) but counts the 252 after-close submissions as the day's trades; they fill Tuesday.
- Momentum's 08-28 equity row is Thursday's equity. The midnight run (00:05Z Fri) stamped its equity row 08-28 and consumed Friday's slot (STAMP-2); Friday's real run advanced nothing for momentum. Friday's close equity was never recorded for the momentum sleeve.
- Captures displaced. Thursday's option capture is stamped 08-28,
Friday's is stamped Saturday 08-29; 08-27 is a hole. Third occurrence of
the
stamp-1UTC-date collision class. Recorded in CAPTURES.md. - pit_snapshots is the real casualty behind the two 🔴 mails. The "skew-snapshot capture failure" alerts of 08-28 and 08-29 were mislabeled: all four option/shortability captures passed QA both days; the failing stream was pit_snapshots — 123 rows (08-27) and 58 rows (08-28) vs a ~890 median, because the snapshotter ran after-hours. 08-26 is missing entirely (no 08-27 run existed to write it), and 08-31's momentum run wrote no snapshot at all despite a green "Point-in-time snapshot" step. capture-qa re-checks the latest file, so the skew job stays red — expect repeat 🔴 mails that carry no new information until a full snapshot lands.
What held
- Kill-switch state advanced on every day a run existed; no safety HALT or REJECT; no halt files. Momentum drawdown −8.4% vs the −20% switch, meanrev −2.7%.
- The out-of-band alarm layer worked as designed: DOWN mails for all four trading-path checks on Friday's 8h-grace breach, matching UPs when the displaced runs finally pinged. thales-skew has been DOWN since 08-28 03:00Z — correctly, since QA is genuinely failing.
- The alert emails themselves (🔴 sleeve-no-run, 🔴 capture failure) fired — in-band alerting survived this outage class, unlike 08-06.
Detector work this implies (queue already holds most of it)
stamp-1(UTC capture date) andstamp-2(trading-path slot consumption) are the root fixes and are already inresearch/queue/open/.- New from this audit: (a) the capture-failure alert should name the
failing stream in its subject; (b) capture-qa should distinguish "latest
file is stale/degraded" from "today's file is missing" instead of
re-flagging the same old file nightly; (c) the pit snapshotter silently
writes nothing when it finds no quotes — a green step with an empty
product, this system's most dangerous shape. These belong to the panel's
queue process (
gap-1/sac-1are adjacent).
The operational question this leaves the owner
Cron times are pinned to a queue-delay assumption that no longer holds. If
scheduled delivery stays this degraded, the configured 14:35–14:45Z crons
need to move hours earlier (e.g. ~11:00Z gives 9h of headroom before the
20:00Z close), or trading needs a scheduler that isn't GitHub's schedule
event. That is a trading-infrastructure decision — the owner's, not a
routine's. Tue 09-01 is the monthly momentum selection day; at Monday's
delay it fires ~20:15Z again and the monthly rebalance prices Tuesday's
close and fills Wednesday's open.