# STAMP-1 — the capture writers use the runner's UTC date as BOTH the trading-day stamp and the idempotency key

status: open · raised panel run 2 (2026-08-07) · migrated 2026-08-24 (body verbatim below) · class: moat / loudness · judgement: **YES** · effort ~0.5 pd

---

**8. STAMP-1 — the capture writers use the runner's UTC date as BOTH the
trading-day stamp and the idempotency key, so a queue-delayed run that crosses
UTC midnight mislabels yesterday's session AND silently consumes today's
slot.** · judgement: **YES** · effort ~0.5 pd · horizon: the code fix is not
time-critical; the 2026-08-07 instance expired at ~20:30Z that day

*Mechanism, code-verified.* `src/thales/cli.py:4472` passes
`today=_dt.date.today()` — the runner's UTC calendar date — into
`snapshot_skew`, with no ET normalisation and no market-session guard;
`cli.py:2057` does the same for `capture_shortability`. That same unvalidated
date is then the idempotency key: `options_skew.py:389-400` builds `done` from
rows whose `date == str(today)` and `:446` skips them; `borrow_flags.py:177-178`
is `if f.exists(): return`.

*It materialised unattended.* Run 31135593381 of `skew-snapshot.yml` was a
**`schedule`** event (`run_attempt: 1`, no human), created
**2026-08-07T00:42:51Z** — 5h43m late, draining the 2026-08-06 runner-outage
backlog — and it succeeded. `data/options_chains/2026-08-07.parquet` carries
`captured_at = 2026-08-07T00:43:45+00:00` (20:43 ET on 08-06, after the 08-06
close and before the 08-07 open) against 20:36:26Z / 20:39:12Z on the
neighbouring days. So 891 names of 08-06 content are stamped
`date: 2026-08-07`, and the 08-07 slot is consumed on four TIME-GATED streams:
`options_skew`, `options_chains`, `options_iv_term`, `shortability`.
`short_volume` is exempt (BACKFILLABLE, carries its own FINRA date).

*Not a clean no-op — measured.* The universe is **903** names
(`data/universe.csv`); **12** are outside `done`. `term_collector` is filled
BEFORE the skew-window check (`options_skew.py:285-293` vs `:294`), and 7 of
those 12 already produced term rows, so `term_rows` is non-empty, the
early-return at `:481-483` does not execute, and the merge at `:548-575`
(`keep="last"`) writes a **two-basis 08-07 term file** — 891 names at the 08-06
basis plus up to 7 at the real 08-07 basis, all with `healed = False`. ≥98.7%
of the 08-07 session is still lost.

*Honest accounting.* One day's content genuinely lost (08-07); one day's
content captured at a degraded after-hours basis and mislabelled (08-06 →
stamped 08-07). **Not "two days lost"** — an earlier draft of this item said
that and it was wrong.

*Materiality — small, stated plainly because the first draft overstated it.*
B4's pinned statistics (`research/2026-08-01_b4_skew_activation_pin.md` §3)
sample **one signal date per ISO week** — "the last capture date carrying ≥400
valid skew rows". Week 32's dates are 08-03 (281, below the floor), 08-04
(897), 08-05 (893), 08-07 (891), so the signal date is 2026-08-07 **in the
counterfactual too**. **S1/S2/S3 lose exactly zero observations**, and the
07-31 → 08-07 chain is 5 td, nowhere near the 10-td drop rule. The only
residual is that week 32's signal content is one session stale — **strictly
conservative**: it attenuates a true IC toward zero and cannot manufacture a
false positive. B1's term pin samples the FIRST capture day of each month
(08-03), unaffected. `shortability` has no pinned consumer. **This is a wrong
label, not a biased statistic.**

What remains real: **2 of 42 accrued skew dates (~4.8%)** are now holed or
mislabelled on a two-month-old stream, and the streams' unregistered
daily-resolution future consumers (single-stock VRP against daily realized
variance; ETB→HTB flip-onset dating, which is the shortability stream's
declared reason to exist) are exactly the ones a day-level error would hurt.
Also: the skew JSONL carries **no `captured_at` field**, so the one stream B4
pins is the one that cannot self-describe the mislabel.

*Prior art.* `TECH_DEBT.md:661-664` TIER 3 "UTC vs ET trading date (MED, low
practical)" predicts the label error verbatim — but names only `daily.py:1288`
/ `vrp_daily.py:308`, describes only the label error, and rates it "low
practical" on a *manual*-run premise, falsified by an unattended scheduled run.
It does not reach `cli.py:4472`, `borrow_flags.py`, or the idempotency
consequence. Nothing in RESEARCH.md / CAPTURES.md / AUDIT.md / the 08-01 panel
report / the B4 pin covers capture-path date stamping.

*The detection half is NOT new — it belongs to GAP-1 (item 4, still open).*
GAP-1 already establishes that `capture_status` computes staleness from
`max(date)` and `capture_qa` only reads `files[-1]`, so an interior hole is
invisible by construction; re-verified for these streams. **This is new
evidence for GAP-1's census, not a second queue row** — with one genuinely new
wrinkle to record there: GAP-1's six holes are all *the run never happened*,
whereas here the run **did** happen and mislabelled its output, so a
date-vs-trading-calendar census would flag the missing 08-06 and would **never**
flag the 08-07 file as wrong. (GAP-1's own table used `options_skew.jsonl` as
its complete-stream control; that control has now itself holed.)

*Correction to the incident record, offered with the item.*
`research/2026-08-06_runner_loss_outage.md:100` says the 08-06 "option-chain,
skew, short-volume and borrow-flag snapshots do not exist". `short_volume` DOES
exist for 08-06 (`data/short_volume/2026-08-06.parquet`, committed by 01792ff)
— FINRA files carry their own data date and the trailing catch-up healed it.
Worth fixing before the weekly audit canonises the cost accounting and before
GAP-1's census starts from a wrong classification.

*Operational trap, recorded because it is destructive and the instinct is
wrong.* `thales snapshot-skew --out data/<anything>.jsonl` resolves
`chains_dir` to `out.parent / "options_chains"` — the SAME
`data/options_chains/2026-08-07.parquet` — whose merge is `keep="last"`. It
would silently overwrite the 08-06 chains capture. Re-dispatching the workflow
is a pure no-op (891 names already `done`). The safe recovery is
`--out /tmp/...`, which sends chains and term to `/tmp` and touches nothing
committed. `_alert.yml`'s hint text ("Re-dispatch TODAY if the market session
has not yet ended") is actively misleading in this failure mode.

*Kill criterion — pre-registered, negative-controlled in both directions.*
On a temp copy only: with the clock frozen at `2026-08-07T00:42:51Z` and the
08-07 artifacts in place, the writer must either stamp `2026-08-06` or refuse
to write and exit non-zero **naming the date** — never silently consume the
08-07 slot. **Negative control:** replay the `captured_at` timestamps of the
last 20 committed capture days (observed band 20:14–20:43Z) and assert the
guard fires on **none** of them; replay a same-session re-run and assert the
idempotent skip still holds. Then shadow 20 trading days: **if the guard fires
on any normally-timed run, or fires zero times while no UTC-midnight crossing
occurred, it is miscalibrated — revert it and record the notches as permanent
rather than tuning the threshold.** Do not fold GAP-1's census/detector work
into this item; duplicating it is how one defect becomes two queue rows.

*Recurrence.* Over the last 30 scheduled runs of this workflow: median lateness
**1.41h**, second-worst **2.01h**, worst **5.71h** (this one), against a
**5.00h** margin from the 19:00Z cron to UTC midnight. A tail event — but one
*correlated with outages*, so it fires precisely when the system is already
degraded.

*CONFIRMED by observation, 2026-08-07 19:55Z.* This item was written as a
prediction before that evening's cron fired; the prediction was pre-registered
in the panel session and then checked. It held on every point. Run
**31213479307** (`schedule`, **success**) ran 19:53:57Z → 19:55:08Z:

| stream | outcome |
|---|---|
| `options_chains/2026-08-07.parquet` | 61,216 rows / 896 underlyings — **61,134 (99.87%) at `captured_at 00:43:45Z`** (the 08-06 basis), only **82 rows (0.13%)** at `19:54:50Z` (the real 08-07 basis) |
| `options_iv_term/2026-08-07.parquet` | 7,842 rows at the 08-06 basis + 33 at 08-07 — the **two-basis file** the review predicted |
| `options_skew.jsonl` | 891 → **896** rows for `2026-08-07`; 5 of the 12 straggler names produced a skew row |
| `shortability/2026-08-07.parquet` | **untouched** — hard skip via `if f.exists(): return` |
| any `2026-08-08` file | none |

`healed` is `False` on every row, so the two bases are separable **only** by
`captured_at` — and nothing reads it. So ~99.9% of the 2026-08-07 option
session is permanently unowned, and the record shows a present, non-stale
`2026-08-07` file on every instrument.

**One new detection hook this observation supplies, which the item did not
have: the run took 71 seconds against ~8 minutes for a normal capture**
(08-05: 8m16s; the 00:42Z run: 8m27s). A ~7x runtime collapse on a
paced-API-call job is a trivially checkable signature of "captured almost
nothing", and no instrument in the repo currently reads it. Worth considering
alongside the ET-normalisation fix — a duration floor would have caught this
class tonight without any date logic at all.

---

## Triage note — 2026-08-31 (annotation only; the body above is untouched)

**THIRD FIRING, previously unrecorded, verified by triage this run: 893 skew
rows are stamped with a Saturday.**

Commit `08386de` "skew: snapshot 2026-08-29", authored **2026-08-29 02:08:21
+0000** — and **2026-08-29 is a Saturday**. It wrote
`data/options_chains/2026-08-29.parquet`,
`data/options_iv_term/2026-08-29.parquet`, and **893 rows into
`data/options_skew.jsonl` stamped `date: 2026-08-29`**. Those rows carry
Friday 08-28's market basis. The stream's recent dates now read: 08-21 (893),
08-24 (893), 08-25 (894), 08-26 (893), 08-28 (894), **08-29 (893)** — with
**2026-08-27 absent entirely**. This is the B4-pinned stream, whose 2027-06
activation checkpoint reads rank-IC by date.

Two details that change the fix, both verified at HEAD `528793d`:

- **One writer in the same commit got it right, and it did not need date
  logic.** `data/short_volume/2026-08-28.parquet` carries its own correct FINRA
  date (BACKFILLABLE — exactly as this row predicted), and `shortability` wrote
  **no** 08-29 file at all, because `borrow_flags.py` guards on
  `is_trading_day(as_of)`. The skew writer has no such guard. **A plain
  trading-day refusal at the skew writer would have caught this instance with
  no ET normalization at all** — worth shipping as leg 0 even if the full
  helper waits.
- **A green tripwire actively sanctions the defect.**
  `tests/test_clock_hermeticity.py` explicitly permits both STAMP forms —
  `today = date.today()` under an `if today is None` fallback, and
  `fn(..., today=date.today())` as "a CLI edge read passed on". It is
  calibrated to test rot, not to session dating, so it reads green on every
  STAMP site. Whoever builds the fix must retune it or the negative control
  will not red.

The rows still carry **no `captured_at` field**, so the mislabel remains
unfalsifiable from the stream itself — which is why each occurrence has had to
be caught by hand, three times now.

**Bundling (triage recommendation, not an edit):** STAMP-1 + STAMP-2 need the
same ET-trading-date helper, which does not exist anywhere in
`utils/calendar.py`. SKW-1 is 0.1 pd inside the same file as this row's skew
site. One PR, ~0.75 pd combined rather than ~1.1 pd apart.

---

## BUILT — 2026-09-02 (together with STAMP-2, as triage recommended)

**Outcome: shipped.** One helper, `thales.utils.calendar.market_session_date`
(the ET calendar date; a pre-open instant belongs to the previous trading
day; a divergence from the runner's UTC date is logged loudly), and every
capture stamp site resolves `today` through it: `thales snapshot-skew`,
`thales snapshot-shortability`, `thales captures` / `capture-qa`. Leg 0 from
the triage note is in: the skew command refuses non-trading days outright,
as the shortability writer already did (a Saturday can no longer be stamped).
`short_volume` untouched (BACKFILLABLE, carries FINRA's own date).

**Route — recorded honestly:** built on the owner's direct instruction in an
interactive session (2026-09-02 self-audit, finding M4), NOT via
`queue/approved/` — the EXQ-1 precedent; the guard denies this open→built
move, so a human merges.

**Kill criterion — met, negative-controlled both ways** (`tests/test_utils/test_market_session_date.py`):
clock frozen at `2026-08-07T00:42:51Z` → the stamp is **2026-08-06** (red before
the helper existed, green after); the 02:01Z Saturday run → **2026-08-28**.
Negative control: the committed capture band (20:14–20:43Z on normal days) and
the 09-01 post-close capture all stamp exactly the UTC date they always did —
the guard fires on none of them. The retuned tripwire the triage note asked
for is the lint in the same file: the writers may no longer read `date.today()`.

**Not done here, recorded:** the "relabel vs quarantine" ruling for the already
mislabelled files (08-07, 08-28, 08-29) is still the owner's (open since the
08-06/07 memo); the run-duration floor the observation suggested is not built
(GAP-1/CQA-2 territory). `captured_at` on the skew JSONL rows is a separate PR.

---

*Provenance recorded by triage 2026-09-07 (annotation only; the record above is untouched): shipped in **PR #132** (`100cf61`, merged 2026-09-03). The capture-store disposition ruling this row opened was executed separately in **PR #152** (`877dce6`, 2026-09-06) via `thales capture-resession` — 239,179 rows relabelled, 85,028 quarantined, 0 dropped (`CAPTURES.md:537-563`).*