Thales
← research journal
raw markdown ↗

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

SKW-3 — the backup firing silently replaces in-session term-structure rows with post-close quotes stamped healed=False: the skew job's second-firing idempotency has a per-stream hole, and capture-QA cannot see it

status: open · raised panel run 19 (2026-09-04) · class: moat integrity (G4 capture layer + G2 loudness — change-review catch, does NOT count toward the frontier cadence) · judgement: YES · effort ~0.25 pd · horizon: recurs EVERY session under the dual-dispatch regime (#134, live since 2026-09-02); each day adds more silently mixed-basis rows

Plain-language summary for an owner reading one paragraph. Since 09-02 the skew capture fires twice per session by design (Mac dispatch at the intended time, GitHub cron as backup), and the second firing is supposed to be a no-op for anything already captured. For most of the data it is. But a name whose skew computation returns nothing (no expiry in the 20–45-day window) gets its term-structure rows written WITHOUT the ledger row that marks it "done" — so the backup firing re-fetches it hours later and its fresh quotes silently REPLACE the in-session capture in the day's term file. Verified in the committed data: on 09-03, 7 names' 19:00Z in-session term rows were overwritten by 17:37 ET post-close quotes; on 09-02, 9 names by 22:22 ET quotes — all stamped healed=False, i.e. labeled as ordinary in-session captures. Row counts don't change, so capture-QA sees nothing, and the workflow's own comment claims the re-run is "an idempotent no-op" — false for exactly this subset. The forward captures are the moat asset; a captured value being silently replaced is worse than a missed one, because a notch at least shows. Mitigant: each firing commits, so the true in-session rows survive in git history and the per-row captured_at is honest — this is recoverable and detectable in the archive, just invisible in the canonical file every consumer reads.

Mechanism — ground truth verified at HEAD this run (src/thales/data/options_skew.py)

  • fetch_symbol_skew collects term rows for ALL usable expiries before the skew-window check (lines 285–293, deliberate — "the term structure exists even on a day when no expiry lands in the [20, 45]-DTE window"), then returns None when compute_skew finds no window expiry (294–296). Chains rows are only collected after that early return (298–311), so a chains name always gets a JSONL row too — the leak is term-only, which is why the committed diffs show chains/shortability untouched while iv_term rewrites.
  • Idempotency keys on the JSONL done set (402–412). A None-skew name has term rows persisted but no JSONL row → never enters done.
  • On the next same-session firing, sym not in done takes the NORMAL fetch path with both collectors live (479–480); the fresh term rows get healed=False + the new captured_at (492–497) and the union at 577–584 (unique(subset=["underlying","expiry"], keep="last")) replaces the earlier in-session rows.
  • The authors knew this exact hazard and guarded the heal path against it (463–467: "letting it reach an already-persisted stream would keep='last'-overwrite good rows with drifted quotes — the basis-honesty concern captured_at/healed exist for"). The same hazard leaks through the not-in-done door, and additionally mislabels the drifted rows healed=False — worse than the heal path it was protecting.
  • Empirical (committed parquets, diffed by key): iv_term/2026-09-03: dde90ae→cd571b2, 8181→8181 rows, zero added, 18 rows across 7 underlyings (ASGN, BK, EA, GTLS, HOLX, JHG, PSTG) replaced captured_at 2026-09-03T19:00:59Z → 21:37:23Z (17:37 ET, post-close), all healed=False. iv_term/2026-09-02: ab655f8→d1a63b9, 7903→7903, 26 rows across 9 underlyings replaced with 2026-09-03T02:22:43Z (22:22 ET) quotes, same mislabel.
  • Why nothing catches it: capture-QA's term checks (row floor vs median, expiries/name, term-date-vs-chains-date) all pass — the row count is byte-for-byte unchanged. The workflow hint (.github/workflows/skew-snapshot.yml:142) states "a same-session re-run is an idempotent no-op for names already written — STAMP-1", which is false for the None-skew subset and would send a responder reading the alert in exactly the wrong direction.
  • This is NOT a STAMP regression: both firings stamped the correct market session (the 02:22Z firing correctly wrote the 09-02 files and caught same-session FINRA short volume — STAMP-1 working as built). The hole is orthogonal: per-stream idempotency, not dating.

Fix shape (propose-only)

Mirror the heal guard on the not-in-done path: pass term_collector=None when sym in term_have (and chain_collector=None when sym in chains_have) so a re-fire fetches only what a stream is actually missing — the second firing then reduces to the pure heal semantics that already exist. Correct the workflow hint sentence in the same change. Cosmetic rider, owner's call: the commit message stamps $(date -u +%F) (UTC) while the content is the market session (d1a63b9 says "snapshot 2026-09-03" but carries 09-02 data) — one-line fix to stamp the session date. Optional remediation: the replaced 09-02/09-03 in-session rows are recoverable from the pre-rewrite commits if the owner wants the canonical files restored; at minimum this row records which rows are post-close basis.

Test design + negative control

Fixture: a symbols list containing one name whose chain yields term rows but no in-window skew expiry; run the snapshot twice into a tmp dir with different mocked quote values. Fixed code → the second run leaves the first run's term rows byte-identical. Negative control (CQA-1 doctrine — revert the guard): the second run must visibly replace the first run's rows with healed=False, reproducing the defect; if it cannot, this finding is wrong and the row moves to dismissed with that recorded. No trading path, no config value, no pinned rule; watching/capture layer only.

Kill criterion — pre-registered

  1. Wrong-diagnosis kill: if the negative control cannot reproduce the overwrite on unfixed code, dismiss this row citing the failed reproduction.
  2. Ruling kill: if the owner rules that latest-wins is the WANTED semantics for the term stream (prefer the latest same-session surface), dismiss with that ruling recorded — but then the healed/captured_at basis contract and the workflow hint must be rewritten to say so, and the reopen condition is the first analysis that needs the in-session surface and cannot get it.

Zero registry rows, zero DSR debt.

Built — recorded 2026-09-06

Shipped (code PR #152). snapshot_skew's normal (not-in-done) path now mirrors the heal guard: chain_collector=None if sym in chains_have and term_collector=None if sym in term_have, so a same-session re-fire fetches only the streams a name is still missing from and the in-session term rows of a None-skew name are never keep="last"-overwritten by post-close quotes. The workflow hint sentence (skew-snapshot.yml) now states the per-stream idempotency honestly, the commit message stamps the market session instead of the runner's UTC date (the cosmetic rider), and — from design review r2 finding 5 the same night — the workflow carries the concurrency: group the other three scheduled state-writers already had.

Kill criterion — met, negative-controlled (tests/test_data/test_options_skew.py::test_second_firing_leaves_in_session_term_rows_byte_identical): a far-expiries-only name (no skew window) captured once, then re-fired with drifted quotes → the term parquet is byte-identical, term_rows == 0, no term_error; with the guard reverted the second run rewrites the file with the drifted atm_iv — the defect reproduces, so the diagnosis holds. The optional remediation (restoring the 09-02/09-03 in-session rows from the pre-rewrite commits) is NOT done here; the rows are recoverable from git and this footer records which dates carry post-close basis for those names.

Route — recorded honestly: built on the owner's direct instruction in the 2026-09-06 interactive session, NOT via queue/approved/; the guard denies the open→built rename, so a human merges.