Short-flow → short-interest nowcast — ONE-SHOT FALSIFICATION (panel A3, 2026-08-01)
Binding document; registered before any computation. The commit sequence
on this branch is the audit trail: this memo commits ALONE, then the
instrument (research/a3_nowcast.py + its synthetic-data tests) commits,
then the analysis runs ONCE and its dated RESULT is appended below the
sentinel line by a final commit. Everything above the sentinel is the frozen
design; the result stands against exactly this text. Choosing any value below
after seeing a predictor↔target correlation would be the gate-shopping this
document exists to prevent; as of this writing, no predictor↔target
computation of any kind has been run — pre-registration work touched
schemas, coverage, flags, and symbol conventions only (disclosed in §8).
Provenance: quant-panel item A3 (short-side specialist; survived 3/3 —
research/2026-08-01_quant_panel_suggestions.md). Data streams registered
2026-07-21 (CAPTURES.md short_volume / short_interest; feasibility log
research/2026-07-21_microcap_data_feasibility.md).
1. Hypothesis and the specialist's prior
The bi-monthly FINRA consolidated short-interest print (total shares held short; settlements the 15th + end-of-month) publishes ~7 business days after settlement — positioning news arrives ~3 weeks stale. IF daily FINRA Reg SHO short-sale volume carried net positioning information, cumulative daily short-volume surprises between two consecutive prints would predict the cross-section of the next print's CHANGE — a nowcast of the print weeks before FINRA publishes it.
The specialist's measurements say it mostly will not, and this one-shot exists to record the answer permanently either way:
- the daily files cover only FINRA-facility-reported (off-exchange) volume — 22–54% venue coverage of the consolidated tape (panel measurement);
- the 57% median short-marked share is market-maker liquidity provision: a market maker shorts to fill a customer BUY and covers intraday, so short-marked volume is often buying pressure, not a bearish bet (panel measurement).
The analysis script prints both caveats next to every result. No returns are touched anywhere in this experiment — predictor and target are both flow/positioning quantities — so the multiple-testing cost to the strategy trial ledger is zero (§7).
2. Inputs (pinned; frozen history tier only)
data/short_volume_history/— 2,002 daily files: 2017-12-29, then 2018-08-01 → 2026-07-20 (the pre-2018-08 range is a known FINRA CDN retention hole). One row per symbol per day:short_volume,total_volume(both are FINRA-facility volume only — the venue caveat). Verified complete against the NYSE calendar on 2018-08-01 → 2026-07-20: zero missing trading days.data/short_interest_history/— 205 settlement partitions, 2017-12-29 → 2026-06-30, consecutive gaps 13–19 calendar days (no missing settlement); columns used:symbol,short_interest,split_flag.- Both stores were bulk-acquired and frozen 2026-07-21/22 (acquire → freeze
→ own). The committed forward-capture dirs (
data/short_volume/,data/short_interest/) are excluded — the input set is exactly the frozen tier the panel item names (2,002 + 205 files). - Calendar: NYSE via
thales.utils.calendar.get_trading_days. short_exempt_volumeis excluded from the predictor (Rule-201-exempt flow is definitionally not ordinary shorting).
3. Window construction (pinned)
- Windows are consecutive settlement pairs (s0, s1) from the history store. Validity guard: 10 ≤ (s1 − s0) ≤ 25 calendar days (all 204 observed pairs pass; the guard exists for a future broken store, not today's).
- Settlement-cycle alignment (mechanical constants, not fitted): a short sale trades T and settles T+2 before 2024-05-28, T+1 from 2024-05-28 (the US T+1 transition). A print as of settlement date s therefore reflects trades through s minus the cycle. Trade-date window: (t0, t1] with t_i = the settlement's anchor trading day (last NYSE trading day ≤ s_i; all 205 settlements ARE trading days) shifted back by lag(s_i) trading days, lag = 2 for s_i < 2024-05-28 else 1.
- Window eligibility: a daily file must exist for EVERY NYSE trading day in (t0, t1] — the predictor is a sum, and a missing day would bias it down mechanically. (Zero files are missing today; first eligible windows begin at the 2018-08 daily-history edge.)
4. Per-name quantities and eligibility (pinned)
For name i in window W = the NYSE trading days in (t0, t1]:
- SV_i = Σ
short_volume, TV_i = Σtotal_volumeover W (a day with no row for the name contributes 0). - presence_i = fraction of days in W with a row for the name.
- Symbol join: daily symbols with "/" removed equal SI
symbolCode(verified: dailyBRK/A↔ SIBRKA). Names failing the join are out. - Eligibility floors (breadth/degeneracy guards, not tuning knobs): presence_i ≥ 0.8 (IPO/delist edge guard); TV_i ≥ 100,000 shares (degenerate-liquidity floor); target present per §5.
Trailing baseline for the surprise candidates, window-length L:
- Trailing days = the L NYSE trading days ending at t0 inclusive (disjoint
from W by construction). Computable at window level iff ≥ 90% of those L
days have daily files; at name level iff the name has rows on ≥ 50% of
the present days and trailing ΣTV > 0. ŝ_i = trailing Σ
short_volume/ trailing Σtotal_volume.
5. Target (pinned)
- ΔSI_i =
short_interest(s1) −short_interest(s0), inner join of the two partitions on symbol. Rows withsplit_flag == "S"in the s1 partition are excluded (share-count rebasing corrupts a share-space change). Duplicate symbols within a partition would be dropped (none exist in the store). - Normalized target: y_i = ΔSI_i / TV_i — the print change expressed as a fraction of the same window FINRA-tape volume that normalizes the predictor. Both sides unit-free in identical units; a predictor that is literally "net new shorting as a fraction of tape volume" maps 1:1 onto y under the hypothesis.
- QA disclosure: FINRA's own
prev_short_interestfield disagrees with the prior partition'sshort_intereston ~39% of rows with median relative difference 0.07% (revision bookkeeping; splits are NOT the driver — 8 of 7,549 mismatches on the sampled pair). The join-based ΔSI is pinned because it guarantees the differenced values are exactly the two settlements that bound the flow-accumulation window.
6. Candidates, fit, verdict (pinned)
Three predictor candidates — the complete set; nothing else will be computed:
- P1
level: SV_i / TV_i — window short-marked share, the zero-knowledge naive mapping (rank-equivalent to cumulative net short flow (2·SV−TV)/TV). - P2
surprise21: SV_i/TV_i − ŝ_i with L = 21. - P3
surprise63: SV_i/TV_i − ŝ_i with L = 63.
Statistic, per window: Spearman rank correlation ρ_w across eligible names between the candidate and y. A window needs ≥ 300 eligible names, else it is skipped (breadth floor; expected name counts run in the thousands). A window where either side is constant is skipped and counted.
- Fit (2018 → 2021): windows with s1 ≤ 2021-12-31 where ALL THREE candidates are computable, each window's ρ computed on the per-window INTERSECTION of the three candidates' eligible names (apples-to-apples). Selected candidate = argmax of median ρ_w across fit windows; tie broken toward fewer parameters (P1 > P2 > P3). Expected fit span ≈ settlements 2018-11 → 2021-12 (~75 windows) given the daily-history edge + L = 63 warm-up.
- Verdict (2022 → 2026, untouched by the fit): windows with 2022-01-01 ≤ s1 ≤ 2026-06-30; the SELECTED candidate only, its own eligibility rules. The verdict statistic is the median ρ_w across verdict windows, compared to 0.20 unrounded. The other two candidates' out-of-sample medians are reported as context and CANNOT alter the verdict.
7. Kill criterion and consequences (pinned; panel language binding)
- Median out-of-sample correlation < 0.20 → the NULL is recorded:
demote the stream permanently — a dated annotation lands in CAPTURES.md
under
short_volumebarring future proposals from citing daily short volume as positioning evidence absent venue-complete data. The capture itself keeps running (insurance + validation rationale unchanged), and the days-to-cover / borrow-screen consumers (panel A4) are explicitly unaffected — they consume the bi-monthly SI print, not daily flow. - ≥ 0.20 → "eligible as timing refinement" ONLY, behind its own future registration (which must then confront the ~T+7-business-day publication lag and the venue-coverage caveat head-on). Not a signal, not a ship, no SHIP-gate standing.
- Either way this file records the answer permanently; the question is closed and may not be relitigated on this data (a venue-complete dataset — e.g. exchange-complete short marking — would be a NEW registration).
- Multiple-testing: zero cost to the strategy trial ledger — no returns are
examined anywhere. The only selection freedom is the disclosed
3-candidate fit, which never sees 2022+ data; the verdict is computed
once. No
results/evaluations/entry is created (this is not a strategy trial).
8. Pre-registration data QA (disclosed; no hypothesis computation)
Performed before this memo froze, limited to schemas/coverage/flags:
store counts and date ranges; settlement gap histogram (13–19 days,
complete); daily-store completeness vs the NYSE calendar (0 missing days
2018-08+); one-day symbol-overlap and class-share conventions (BRK/A ↔
BRKA); split_flag/revision_flag value sets; prev-field vs prior-file
mismatch magnitudes (§5); ADV-field zeros (2,007/20,251 rows on the sampled
partition — FINRA ADV is therefore NOT used anywhere in this design).
No short-volume quantity was ever joined to a short-interest change before
this memo's commit.
9. Instrument
research/a3_nowcast.py implements exactly this memo; where they could
disagree, the memo wins and the discrepancy is a bug to fix BEFORE the run.
Its tests validate the instrument on synthetic stores only: a planted
flow→ΔSI effect must be detected (ρ ≈ 1), an independent target must read
null, the settlement-lag constants, split exclusion, symbol-join
normalization, coverage floor, and fit/verdict separation are each
asserted. The real stores are touched exactly once, by the registered run.
SENTINEL — design frozen above this line. Result appended below by a later commit; nothing above may change.
RESULT — 2026-08-01 (the one registered run)
Commit order honored: memo 82b176e → instrument + synthetic tests
c916658 → this append. Single invocation of research/a3_nowcast.py
(polars 1.38.1, scipy 1.17.1) against the frozen tier exactly as pinned
(2,002 daily files 2017-12-29..2026-07-20; 205 partitions
2017-12-29..2026-06-30). Per-window audit table:
research/a3_nowcast_windows.json (184 rows, committed with this append).
Fit era — 76 windows, s1 2018-11-15 → 2021-12-31, median 6,019 names/window (skips: 15 windows on the known 2018-01→07 daily CDN hole, 5 on the 63-td baseline warm-up):
| candidate | median fit ρ |
|---|---|
| level | +0.1763 |
| surprise21 (selected) | +0.2305 |
| surprise63 | +0.2237 |
Verdict era (untouched) — candidate surprise21, 108 windows, s1
2022-01-14 → 2026-06-30, median 7,146 names/window:
- median OOS Spearman ρ = +0.2414 — IQR [+0.2116, +0.2562], 108/108 windows positive, per-year medians 2022 +0.2129, 2023 +0.2108, 2024 +0.2492, 2025 +0.2564, 2026 +0.2716.
- Context (non-binding): level +0.1747, surprise63 +0.2301 OOS.
- Short-share echo among eligible names: ~0.44 (fit) / ~0.47 (verdict) window medians — consistent with the panel's 57% consolidated-universe figure.
VERDICT per the pinned criterion (§7): +0.2414 ≥ 0.20 → NOT the null. The permanent-demotion branch does not fire; no proposal bar is created. The stream earns exactly and only what §7 grants: "eligible as timing refinement", behind its own future registration — no signal standing, no SHIP-gate standing, nothing runs on this result. Any future registration must confront head-on: (a) 22–54% venue coverage, (b) the market-maker majority of short-marked flow, (c) the ~T+7-business-day publication lag of the print being nowcast.
Honest reading, bounded by the printed caveats. The fit-to-OOS stability (0.2305 → 0.2414, slightly HIGHER out of sample, monotone-rising by year) says the trace is real and not an overfit signature — but ρ ≈ 0.24 means ρ² ≈ 0.06: ~94% of the cross-sectional variance in print changes is NOT in daily short flow. The specialist's mechanism critique stands materially intact — most short-marked volume is liquidity provision; a minority positioning component exists and survives the venue-coverage haircut. Both prints are now on the record: the effect is too weak to call daily short volume "positioning", and too stable to call it nothing.
The question is CLOSED on this data (§7): re-opening requires venue-complete data under a NEW registration.