Thales
← research journal
raw markdown ↗

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

CQA-2 — capture-qa reads only the latest file, so a missing today-file fails nothing and a failing stream mails under another stream's name

status: open · raised panel run 16 (2026-09-01), executing the 2026-08-31 daily-audit memo's detector hand-off (items a+b, folded by adversarial review) · class: instrument-calibration / loudness (capture QA layer) · judgement: YES · effort ~0.25–0.4 pd · horizon: live now — the memo predicts repeat mislabeled 🔴 mails until a full pit snapshot lands, and the trailing-hole blindness holds for every expect-every-day stream

Plain-language summary for an owner reading one paragraph. The nightly capture quality check only ever looks at the newest file a stream has. Two consequences, both observed last week: when today's file is missing entirely, nothing fails — the check re-reads yesterday's file, so the one condition that means "we are losing a day right now" is invisible to the one instrument that can red; and because the check for ALL streams runs inside the skew-snapshot job, the failure mails arrive as "🔴 skew-snapshot capture" even when the failing stream is pit_snapshots, owned by a different workflow — which is exactly how the 08-28/08-29 alerts mislabeled themselves. The fix: make "today's file is missing" a distinct failing condition naming the stream and date, and put the failing stream names in the alert subject.

Mechanism — verified at HEAD 29c6f23

  • capture_qa reads files[-1] per stream only (src/thales/data/captures.py:810-834). "Today's file missing" is not a failing condition anywhere: staleness (capture_status) is report-only by design (exit 0), and the interior-gap detector capture_gaps windows "up to the stream's OWN latest capture (never past it — a stream that is merely stale is STALE, which staleness already reports)" (its own comment, captures.py:357-363). That deliberate hand-off cedes the trailing hole to a channel that fails nothing — the hand-off with no failing endpoint is this row's defect, cited here so the row cannot be read as GAP-1 relitigation (GAP-1's narrowed scope is the full-history census + the window ruling).
  • The alert subject is workflow-static: _alert.yml builds Subject: 🔴 {label} {result} from a workflow-level env (.github/workflows/_alert.yml:85), and skew-snapshot.yml:103 runs thales capture-qa across all registered streams — so a pit_snapshots failure (owned by paper-trading.yml) is structurally guaranteed to mail as a skew failure. Per-stream ❌ lines exist only in the Actions log.

Struck by adversarial review, recorded so it stays struck: the memo's complaint that the same degraded file "re-fires nightly with no new information" is NOT part of this row. A standing red until fixed is correct fail-closed behavior, and the tempting remedy — an acknowledged-degradation ledger — would be a second ACK surface and a mechanism for making reds quiet, the opposite of this shop's doctrine. The observed harm was misattribution, not repetition.

Fix shape (propose-only)

  1. Missing-today check: for streams registered as expect-every-trading-day, a trading day with no file (after the stream's normal publication lag — pit is legitimately T-1) is a distinct capture_qa failure naming the stream and the missing date.
  2. Subject attribution: capture-qa emits the failing stream names (e.g. to a step output/file) and the alert includes them in the subject — "🔴 capture-qa: pit_snapshots" instead of the owning workflow's static label.

Test design + kill criterion (pre-registered, negative-controlled)

  • Fixture with today's file absent on a trading day → a distinct "missing" failure naming stream + date; revert → red.
  • Lag negative control (required): a stream at T-1 within its normal registered publication lag must NOT red — without this the check manufactures a nightly false positive by construction, the alert-fatigue class this shop keeps paying for.
  • Subject fixture: a forced pit_snapshots QA failure inside the skew job produces an alert whose subject names pit_snapshots; revert the wiring → only its test reds.
  • Falsifier: if the missing-today check produces ≥1 false red on a legitimate holiday/lag day within 20 shadow trading days, the lag model is wrong — record and re-derive per-stream lags from the stream's own history; do not whitelist dates ad hoc.

Data plan

None — owned state and static code; rides the existing capture-qa cron.

Cap note

Ranked second of this run's two rows (PIT-1 first: it stops permanent data loss; this row stops misattribution and blindness, nothing is destroyed). If the 12-cap must hold hard after SKW-1's open→built move lands, this is the row to hold at the gate.