Thales
← research journal
Aug 1, 2026raw markdown ↗

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

Short-flow → short-interest nowcast — ONE-SHOT FALSIFICATION (panel A3, 2026-08-01)

Binding document; registered before any computation. The commit sequence on this branch is the audit trail: this memo commits ALONE, then the instrument (research/a3_nowcast.py + its synthetic-data tests) commits, then the analysis runs ONCE and its dated RESULT is appended below the sentinel line by a final commit. Everything above the sentinel is the frozen design; the result stands against exactly this text. Choosing any value below after seeing a predictor↔target correlation would be the gate-shopping this document exists to prevent; as of this writing, no predictor↔target computation of any kind has been run — pre-registration work touched schemas, coverage, flags, and symbol conventions only (disclosed in §8).

Provenance: quant-panel item A3 (short-side specialist; survived 3/3 — research/2026-08-01_quant_panel_suggestions.md). Data streams registered 2026-07-21 (CAPTURES.md short_volume / short_interest; feasibility log research/2026-07-21_microcap_data_feasibility.md).

1. Hypothesis and the specialist's prior

The bi-monthly FINRA consolidated short-interest print (total shares held short; settlements the 15th + end-of-month) publishes ~7 business days after settlement — positioning news arrives ~3 weeks stale. IF daily FINRA Reg SHO short-sale volume carried net positioning information, cumulative daily short-volume surprises between two consecutive prints would predict the cross-section of the next print's CHANGE — a nowcast of the print weeks before FINRA publishes it.

The specialist's measurements say it mostly will not, and this one-shot exists to record the answer permanently either way:

  • the daily files cover only FINRA-facility-reported (off-exchange) volume — 22–54% venue coverage of the consolidated tape (panel measurement);
  • the 57% median short-marked share is market-maker liquidity provision: a market maker shorts to fill a customer BUY and covers intraday, so short-marked volume is often buying pressure, not a bearish bet (panel measurement).

The analysis script prints both caveats next to every result. No returns are touched anywhere in this experiment — predictor and target are both flow/positioning quantities — so the multiple-testing cost to the strategy trial ledger is zero (§7).

2. Inputs (pinned; frozen history tier only)

  • data/short_volume_history/2,002 daily files: 2017-12-29, then 2018-08-01 → 2026-07-20 (the pre-2018-08 range is a known FINRA CDN retention hole). One row per symbol per day: short_volume, total_volume (both are FINRA-facility volume only — the venue caveat). Verified complete against the NYSE calendar on 2018-08-01 → 2026-07-20: zero missing trading days.
  • data/short_interest_history/205 settlement partitions, 2017-12-29 → 2026-06-30, consecutive gaps 13–19 calendar days (no missing settlement); columns used: symbol, short_interest, split_flag.
  • Both stores were bulk-acquired and frozen 2026-07-21/22 (acquire → freeze → own). The committed forward-capture dirs (data/short_volume/, data/short_interest/) are excluded — the input set is exactly the frozen tier the panel item names (2,002 + 205 files).
  • Calendar: NYSE via thales.utils.calendar.get_trading_days.
  • short_exempt_volume is excluded from the predictor (Rule-201-exempt flow is definitionally not ordinary shorting).

3. Window construction (pinned)

  • Windows are consecutive settlement pairs (s0, s1) from the history store. Validity guard: 10 ≤ (s1 − s0) ≤ 25 calendar days (all 204 observed pairs pass; the guard exists for a future broken store, not today's).
  • Settlement-cycle alignment (mechanical constants, not fitted): a short sale trades T and settles T+2 before 2024-05-28, T+1 from 2024-05-28 (the US T+1 transition). A print as of settlement date s therefore reflects trades through s minus the cycle. Trade-date window: (t0, t1] with t_i = the settlement's anchor trading day (last NYSE trading day ≤ s_i; all 205 settlements ARE trading days) shifted back by lag(s_i) trading days, lag = 2 for s_i < 2024-05-28 else 1.
  • Window eligibility: a daily file must exist for EVERY NYSE trading day in (t0, t1] — the predictor is a sum, and a missing day would bias it down mechanically. (Zero files are missing today; first eligible windows begin at the 2018-08 daily-history edge.)

4. Per-name quantities and eligibility (pinned)

For name i in window W = the NYSE trading days in (t0, t1]:

  • SV_i = Σ short_volume, TV_i = Σ total_volume over W (a day with no row for the name contributes 0).
  • presence_i = fraction of days in W with a row for the name.
  • Symbol join: daily symbols with "/" removed equal SI symbolCode (verified: daily BRK/A ↔ SI BRKA). Names failing the join are out.
  • Eligibility floors (breadth/degeneracy guards, not tuning knobs): presence_i ≥ 0.8 (IPO/delist edge guard); TV_i ≥ 100,000 shares (degenerate-liquidity floor); target present per §5.

Trailing baseline for the surprise candidates, window-length L:

  • Trailing days = the L NYSE trading days ending at t0 inclusive (disjoint from W by construction). Computable at window level iff ≥ 90% of those L days have daily files; at name level iff the name has rows on ≥ 50% of the present days and trailing ΣTV > 0. ŝ_i = trailing Σshort_volume / trailing Σtotal_volume.

5. Target (pinned)

  • ΔSI_i = short_interest(s1) − short_interest(s0), inner join of the two partitions on symbol. Rows with split_flag == "S" in the s1 partition are excluded (share-count rebasing corrupts a share-space change). Duplicate symbols within a partition would be dropped (none exist in the store).
  • Normalized target: y_i = ΔSI_i / TV_i — the print change expressed as a fraction of the same window FINRA-tape volume that normalizes the predictor. Both sides unit-free in identical units; a predictor that is literally "net new shorting as a fraction of tape volume" maps 1:1 onto y under the hypothesis.
  • QA disclosure: FINRA's own prev_short_interest field disagrees with the prior partition's short_interest on ~39% of rows with median relative difference 0.07% (revision bookkeeping; splits are NOT the driver — 8 of 7,549 mismatches on the sampled pair). The join-based ΔSI is pinned because it guarantees the differenced values are exactly the two settlements that bound the flow-accumulation window.

6. Candidates, fit, verdict (pinned)

Three predictor candidates — the complete set; nothing else will be computed:

  • P1 level: SV_i / TV_i — window short-marked share, the zero-knowledge naive mapping (rank-equivalent to cumulative net short flow (2·SV−TV)/TV).
  • P2 surprise21: SV_i/TV_i − ŝ_i with L = 21.
  • P3 surprise63: SV_i/TV_i − ŝ_i with L = 63.

Statistic, per window: Spearman rank correlation ρ_w across eligible names between the candidate and y. A window needs ≥ 300 eligible names, else it is skipped (breadth floor; expected name counts run in the thousands). A window where either side is constant is skipped and counted.

  • Fit (2018 → 2021): windows with s1 ≤ 2021-12-31 where ALL THREE candidates are computable, each window's ρ computed on the per-window INTERSECTION of the three candidates' eligible names (apples-to-apples). Selected candidate = argmax of median ρ_w across fit windows; tie broken toward fewer parameters (P1 > P2 > P3). Expected fit span ≈ settlements 2018-11 → 2021-12 (~75 windows) given the daily-history edge + L = 63 warm-up.
  • Verdict (2022 → 2026, untouched by the fit): windows with 2022-01-01 ≤ s1 ≤ 2026-06-30; the SELECTED candidate only, its own eligibility rules. The verdict statistic is the median ρ_w across verdict windows, compared to 0.20 unrounded. The other two candidates' out-of-sample medians are reported as context and CANNOT alter the verdict.

7. Kill criterion and consequences (pinned; panel language binding)

  • Median out-of-sample correlation < 0.20 → the NULL is recorded: demote the stream permanently — a dated annotation lands in CAPTURES.md under short_volume barring future proposals from citing daily short volume as positioning evidence absent venue-complete data. The capture itself keeps running (insurance + validation rationale unchanged), and the days-to-cover / borrow-screen consumers (panel A4) are explicitly unaffected — they consume the bi-monthly SI print, not daily flow.
  • ≥ 0.20 → "eligible as timing refinement" ONLY, behind its own future registration (which must then confront the ~T+7-business-day publication lag and the venue-coverage caveat head-on). Not a signal, not a ship, no SHIP-gate standing.
  • Either way this file records the answer permanently; the question is closed and may not be relitigated on this data (a venue-complete dataset — e.g. exchange-complete short marking — would be a NEW registration).
  • Multiple-testing: zero cost to the strategy trial ledger — no returns are examined anywhere. The only selection freedom is the disclosed 3-candidate fit, which never sees 2022+ data; the verdict is computed once. No results/evaluations/ entry is created (this is not a strategy trial).

8. Pre-registration data QA (disclosed; no hypothesis computation)

Performed before this memo froze, limited to schemas/coverage/flags: store counts and date ranges; settlement gap histogram (13–19 days, complete); daily-store completeness vs the NYSE calendar (0 missing days 2018-08+); one-day symbol-overlap and class-share conventions (BRK/ABRKA); split_flag/revision_flag value sets; prev-field vs prior-file mismatch magnitudes (§5); ADV-field zeros (2,007/20,251 rows on the sampled partition — FINRA ADV is therefore NOT used anywhere in this design). No short-volume quantity was ever joined to a short-interest change before this memo's commit.

9. Instrument

research/a3_nowcast.py implements exactly this memo; where they could disagree, the memo wins and the discrepancy is a bug to fix BEFORE the run. Its tests validate the instrument on synthetic stores only: a planted flow→ΔSI effect must be detected (ρ ≈ 1), an independent target must read null, the settlement-lag constants, split exclusion, symbol-join normalization, coverage floor, and fit/verdict separation are each asserted. The real stores are touched exactly once, by the registered run.


SENTINEL — design frozen above this line. Result appended below by a later commit; nothing above may change.

RESULT — 2026-08-01 (the one registered run)

Commit order honored: memo 82b176e → instrument + synthetic tests c916658 → this append. Single invocation of research/a3_nowcast.py (polars 1.38.1, scipy 1.17.1) against the frozen tier exactly as pinned (2,002 daily files 2017-12-29..2026-07-20; 205 partitions 2017-12-29..2026-06-30). Per-window audit table: research/a3_nowcast_windows.json (184 rows, committed with this append).

Fit era — 76 windows, s1 2018-11-15 → 2021-12-31, median 6,019 names/window (skips: 15 windows on the known 2018-01→07 daily CDN hole, 5 on the 63-td baseline warm-up):

candidatemedian fit ρ
level+0.1763
surprise21 (selected)+0.2305
surprise63+0.2237

Verdict era (untouched) — candidate surprise21, 108 windows, s1 2022-01-14 → 2026-06-30, median 7,146 names/window:

  • median OOS Spearman ρ = +0.2414 — IQR [+0.2116, +0.2562], 108/108 windows positive, per-year medians 2022 +0.2129, 2023 +0.2108, 2024 +0.2492, 2025 +0.2564, 2026 +0.2716.
  • Context (non-binding): level +0.1747, surprise63 +0.2301 OOS.
  • Short-share echo among eligible names: ~0.44 (fit) / ~0.47 (verdict) window medians — consistent with the panel's 57% consolidated-universe figure.

VERDICT per the pinned criterion (§7): +0.2414 ≥ 0.20 → NOT the null. The permanent-demotion branch does not fire; no proposal bar is created. The stream earns exactly and only what §7 grants: "eligible as timing refinement", behind its own future registration — no signal standing, no SHIP-gate standing, nothing runs on this result. Any future registration must confront head-on: (a) 22–54% venue coverage, (b) the market-maker majority of short-marked flow, (c) the ~T+7-business-day publication lag of the print being nowcast.

Honest reading, bounded by the printed caveats. The fit-to-OOS stability (0.2305 → 0.2414, slightly HIGHER out of sample, monotone-rising by year) says the trace is real and not an overfit signature — but ρ ≈ 0.24 means ρ² ≈ 0.06: ~94% of the cross-sectional variance in print changes is NOT in daily short flow. The specialist's mechanism critique stands materially intact — most short-marked volume is liquidity provision; a minority positioning component exists and survives the venue-coverage haircut. Both prints are now on the record: the effect is too weak to call daily short volume "positioning", and too stable to call it nothing.

The question is CLOSED on this data (§7): re-opening requires venue-complete data under a NEW registration.