SKW-4 — the skew store serves 2,068 rows stamped on non-sessions (two NYSE holidays and a Saturday) through a reader with no trading-day filter, and per-name coverage is unmeasured: GHC and NVR have zero rows in 62 capture days and no instrument would say so
status: open · raised 2026-09-05 (external design review r2, finding 4 + a near-miss; maintainer-reproduced exactly against get_trading_days) · class: moat integrity (G4 capture layer, consumer side — the twin of GAP-1's missing days: days that should not exist) · judgement: YES for the reader filter and the coverage census (mechanical); the store disposition is the owner's standing STAMP-1 ruling · effort ~0.25 pd + the ruling · horizon: manifests on the first study that groups the stream by date; the per-name holes are permanent and grow by one notch per name per session
Plain-language summary for an owner reading one paragraph. STAMP-1 fixed the writer: the skew capture now refuses to run on a day the market is closed. But 2,068 rows written before that fix — on Juneteenth, on the observed Independence Day holiday, and on a Saturday — are still in the canonical file, and the function every study uses to read the file hands them back unfiltered. To any analysis that groups by date they look like three extra trading sessions carrying the previous session's stale quotes, inside the dataset this project calls its moat and publishes. Separately, the capture's only per-day quality check is "did roughly the usual number of rows arrive"; a name that never arrives is 0.1% of that count and invisible. Two names in the universe, GHC and NVR, have never contributed a row in 62 capture days, and they are not the two class-share names SKW-2 already covers. The fix has three parts: filter on read (one line), decide what to do with the 2,068 rows (the owner's already-open STAMP-1 ruling — quarantine recommended), and add a per-name coverage census to capture-QA.
Mechanism — verified at HEAD cd571b2, data at 3779020 (50,438 rows, 62 dates, 0 duplicate (date, symbol) keys)
Leg 1 — non-session rows served.
| date | rows | why it is not a session |
|---|---|---|
| 2026-06-19 | 892 | NYSE holiday (Juneteenth) — the cron is Mon–Fri, not calendar-aware |
| 2026-07-03 | 283 | NYSE holiday (Independence Day observed) |
| 2026-08-29 | 893 | Saturday — the STAMP-1 incident (a 02:01Z Saturday run stamped the runner's UTC date) |
Re-derived with thales.utils.calendar.get_trading_days; total 2,068 of
50,438. The writer was fixed by STAMP-1 (cli.py:4592-4598 refuses
non-trading days and stamps the market session). load_skew
(options_skew.py:316-344) returns every row verbatim; nothing on the read
path re-checks. Because the (date, symbol) keys are unique, these are not
double-captures — they are phantom sessions: Alpaca serves the previous
close's chain on a closed day, so the rows carry drifted stale quotes under a
date on which nothing was observed. This is the classic fix-sweep failure the
reviewer named: the fix landed where the bug was reported (the writer), not
everywhere it lives (the stored rows, every reader that trusts them).
Leg 2 — per-name coverage unmeasured.
| symbol | capture days present (of 62) |
|---|---|
| GHC | 0 |
| NVR | 0 |
| MASI | 1 |
| ASGN | 18 |
| HOLX | 18 |
| PSTG | 19 |
| BK | 32 |
| CTRA | 35 |
capture_qa's per-day skew test is total row count against a trailing median
(captures.py:610-629); one permanently absent name is ~0.1% of that count.
SKW-2 covers exactly two names (BRK-B, BF-B) for a known cause (dash
symbology). GHC and NVR are dash-free and outside its scope, and their cause
is unknown — chain absent at the venue, fetch failing, or no expiry in the
[20, 45]-DTE window every day (the SKW-3 None-skew shape). Determine, don't
guess: the census is the instrument; the cause is its first output.
Fix shape (propose-only)
- Reader filter.
load_skew(..., sessions_only=True)as the default: drop rows whose date failsis_trading_day, log the count dropped. Apply the same to the term and chains loaders if they share the path. One line plus a log line; safe regardless of the ruling in 2. - Store disposition — the owner's ruling, not the panel's. The STAMP-1
built-row footer records the "relabel vs quarantine" ruling for the
already-mislabelled captures (08-07, 08-28, 08-29) as still open since
the 08-06/07 memo. These three JSONL dates are the same ruling.
Recommendation: quarantine, not relabel — the rows carry quotes that
drifted from a session nobody observed, so relabelling would assert an
observation that did not happen. Mechanism: a
thales captures quarantine --non-sessioncommand that moves the rows to a sibling*_quarantine.jsonlwith a CAPTURES.md note — never a text edit of committed data (this shop does not hand-edit history; 2026-08-29 memo). - Coverage census in
capture_qa: names with zero rows over the trailing N sessions, listed by name, plus a per-name presence ratio in the report. Either re-scope SKW-2 from "two dash-class names" to this census, or keep SKW-2 as the symbology fix and let this row own the census — triage's call; the two must not both claim it.
Test design + negative control (CQA-1 doctrine)
load_skewover a fixture containing a Saturday-dated row → excluded by default, included withsessions_only=False; revert → red.capture_qaover a fixture where one name is absent for 5 sessions → named in the report; revert → silent (today's behaviour).- Quarantine command over a fixture with one holiday date → rows moved, key count conserved across the two files, CAPTURES.md note asserted.
Kill criterion — pre-registered
If the owner rules that non-session rows are wanted as "last-known-surface"
observations, leg 1 becomes a session: bool column rather than a filter and
leg 2 becomes "keep, flagged"; the row records the ruling and closes on the
column. Leg 3 has no kill — an unmeasured coverage is a defect by the
charter's own G4 wording regardless of what the census finds.
Cap note for triage
Fold partner SKW-2 (leg 3) / the STAMP-1 ruling (leg 2). Never GAP-1 — it owns missing days and the G4 gauge claim; this row is the mirror axis (phantom days, absent names).
Built — recorded 2026-09-06; legs 1–2 shipped, leg 3 re-routed
Leg 1 — reader filter (code PR #152). load_skew(sessions_only=True) is
the default: rows stamped on non-session dates are dropped on read and the
count is logged.
Leg 2 — the store, under the owner's ruling (2026-09-06). The ruling
turned out to be TWO rulings, because the 2,068 rows are two classes — see
the verification memo and the CAPTURES.md "Data-disposition rulings" entry
written by thales capture-resession --apply:
- Holiday re-serves (06-19: 892 rows; 07-03: 283; plus their chains
files): no session was observed and the previous session already had its
own rows → quarantined to
data/quarantine/, never deleted. - STAMP-class mislabels (the 08-29-stamped 893 rows, and — found while
verifying — the 08-07- and 08-28-stamped ones): genuine post-close
observations of the previous session, stamped with the runner's UTC date →
re-sessioned per row by
captured_at(08-07→08-06 for the 00:43Z firing, 08-28→08-27, 08-29→08-28; the 5 names genuinely captured in-session on 08-07 stay), acrossoptions_skew,options_chains,options_iv_termandshortability. Two true holes remain and are acknowledged incaptures.ACKNOWLEDGED_GAPS: shortability 08-07 and 08-28, whose genuine states were skipped by the writer's file-idempotency.pit_snapshotsdeliberately untouched (PIT-1; different date semantics).
Leg 3 — per-name coverage census: NOT built here. Re-routed to SKW-2's re-scope (the row's own either/or): GHC/NVR at 0/62 capture days and the low-coverage names are its census's first output; the mechanism stays "determine, don't guess".
Route — recorded honestly: built on the owner's direct instruction in the
2026-09-06 interactive session, NOT via queue/approved/; the guard denies
the open→built rename, so a human merges.