# Short-flow → short-interest nowcast — ONE-SHOT FALSIFICATION (panel A3, 2026-08-01)

**Binding document; registered before any computation.** The commit sequence
on this branch is the audit trail: this memo commits ALONE, then the
instrument (`research/a3_nowcast.py` + its synthetic-data tests) commits,
then the analysis runs ONCE and its dated RESULT is appended below the
sentinel line by a final commit. Everything above the sentinel is the frozen
design; the result stands against exactly this text. Choosing any value below
after seeing a predictor↔target correlation would be the gate-shopping this
document exists to prevent; as of this writing, **no predictor↔target
computation of any kind has been run** — pre-registration work touched
schemas, coverage, flags, and symbol conventions only (disclosed in §8).

Provenance: quant-panel item **A3** (short-side specialist; survived 3/3 —
`research/2026-08-01_quant_panel_suggestions.md`). Data streams registered
2026-07-21 (CAPTURES.md `short_volume` / `short_interest`; feasibility log
`research/2026-07-21_microcap_data_feasibility.md`).

## 1. Hypothesis and the specialist's prior

The bi-monthly FINRA consolidated short-interest print (total shares held
short; settlements the 15th + end-of-month) publishes ~7 business days after
settlement — positioning news arrives ~3 weeks stale. **IF** daily FINRA
Reg SHO short-sale volume carried net positioning information, cumulative
daily short-volume *surprises* between two consecutive prints would predict
the cross-section of the next print's CHANGE — a nowcast of the print weeks
before FINRA publishes it.

The specialist's measurements say it mostly will not, and this one-shot
exists to record the answer permanently either way:

- the daily files cover only FINRA-facility-reported (off-exchange) volume —
  **22–54% venue coverage** of the consolidated tape (panel measurement);
- the **57% median short-marked share** is market-maker liquidity provision:
  a market maker shorts to fill a customer BUY and covers intraday, so
  short-marked volume is often *buying* pressure, not a bearish bet (panel
  measurement).

The analysis script prints both caveats next to every result. No returns are
touched anywhere in this experiment — predictor and target are both
flow/positioning quantities — so the multiple-testing cost to the strategy
trial ledger is **zero** (§7).

## 2. Inputs (pinned; frozen history tier only)

- `data/short_volume_history/` — **2,002 daily files**: 2017-12-29, then
  2018-08-01 → 2026-07-20 (the pre-2018-08 range is a known FINRA CDN
  retention hole). One row per symbol per day: `short_volume`,
  `total_volume` (both are FINRA-facility volume only — the venue caveat).
  Verified complete against the NYSE calendar on 2018-08-01 → 2026-07-20:
  zero missing trading days.
- `data/short_interest_history/` — **205 settlement partitions**,
  2017-12-29 → 2026-06-30, consecutive gaps 13–19 calendar days (no missing
  settlement); columns used: `symbol`, `short_interest`, `split_flag`.
- Both stores were bulk-acquired and frozen 2026-07-21/22 (acquire → freeze
  → own). The committed forward-capture dirs (`data/short_volume/`,
  `data/short_interest/`) are **excluded** — the input set is exactly the
  frozen tier the panel item names (2,002 + 205 files).
- Calendar: NYSE via `thales.utils.calendar.get_trading_days`.
- `short_exempt_volume` is excluded from the predictor (Rule-201-exempt
  flow is definitionally not ordinary shorting).

## 3. Window construction (pinned)

- Windows are consecutive settlement pairs (s0, s1) from the history store.
  Validity guard: 10 ≤ (s1 − s0) ≤ 25 calendar days (all 204 observed pairs
  pass; the guard exists for a future broken store, not today's).
- **Settlement-cycle alignment** (mechanical constants, not fitted): a short
  sale trades T and settles T+2 before 2024-05-28, T+1 from 2024-05-28 (the
  US T+1 transition). A print as of settlement date s therefore reflects
  trades through s minus the cycle. Trade-date window: **(t0, t1]** with
  t_i = the settlement's anchor trading day (last NYSE trading day ≤ s_i;
  all 205 settlements ARE trading days) shifted back by lag(s_i) trading
  days, lag = 2 for s_i < 2024-05-28 else 1.
- Window eligibility: a daily file must exist for EVERY NYSE trading day in
  (t0, t1] — the predictor is a sum, and a missing day would bias it down
  mechanically. (Zero files are missing today; first eligible windows begin
  at the 2018-08 daily-history edge.)

## 4. Per-name quantities and eligibility (pinned)

For name i in window W = the NYSE trading days in (t0, t1]:

- SV_i = Σ `short_volume`, TV_i = Σ `total_volume` over W (a day with no
  row for the name contributes 0).
- presence_i = fraction of days in W with a row for the name.
- **Symbol join**: daily symbols with "/" removed equal SI `symbolCode`
  (verified: daily `BRK/A` ↔ SI `BRKA`). Names failing the join are out.
- Eligibility floors (breadth/degeneracy guards, not tuning knobs):
  presence_i ≥ 0.8 (IPO/delist edge guard); TV_i ≥ 100,000 shares
  (degenerate-liquidity floor); target present per §5.

Trailing baseline for the surprise candidates, window-length L:

- Trailing days = the L NYSE trading days ending at t0 inclusive (disjoint
  from W by construction). Computable at window level iff ≥ 90% of those L
  days have daily files; at name level iff the name has rows on ≥ 50% of
  the present days and trailing ΣTV > 0. ŝ_i = trailing Σ`short_volume` /
  trailing Σ`total_volume`.

## 5. Target (pinned)

- ΔSI_i = `short_interest`(s1) − `short_interest`(s0), inner join of the
  two partitions on symbol. Rows with `split_flag == "S"` in the s1
  partition are excluded (share-count rebasing corrupts a share-space
  change). Duplicate symbols within a partition would be dropped (none
  exist in the store).
- Normalized target: **y_i = ΔSI_i / TV_i** — the print change expressed as
  a fraction of the same window FINRA-tape volume that normalizes the
  predictor. Both sides unit-free in identical units; a predictor that is
  literally "net new shorting as a fraction of tape volume" maps 1:1 onto y
  under the hypothesis.
- QA disclosure: FINRA's own `prev_short_interest` field disagrees with the
  prior partition's `short_interest` on ~39% of rows with median relative
  difference 0.07% (revision bookkeeping; splits are NOT the driver — 8 of
  7,549 mismatches on the sampled pair). The join-based ΔSI is pinned
  because it guarantees the differenced values are exactly the two
  settlements that bound the flow-accumulation window.

## 6. Candidates, fit, verdict (pinned)

Three predictor candidates — the complete set; nothing else will be
computed:

- **P1 `level`**: SV_i / TV_i — window short-marked share, the
  zero-knowledge naive mapping (rank-equivalent to cumulative net short
  flow (2·SV−TV)/TV).
- **P2 `surprise21`**: SV_i/TV_i − ŝ_i with L = 21.
- **P3 `surprise63`**: SV_i/TV_i − ŝ_i with L = 63.

Statistic, per window: **Spearman rank correlation ρ_w** across eligible
names between the candidate and y. A window needs ≥ 300 eligible names,
else it is skipped (breadth floor; expected name counts run in the
thousands). A window where either side is constant is skipped and counted.

- **Fit (2018 → 2021)**: windows with s1 ≤ 2021-12-31 where ALL THREE
  candidates are computable, each window's ρ computed on the per-window
  INTERSECTION of the three candidates' eligible names (apples-to-apples).
  Selected candidate = argmax of median ρ_w across fit windows; tie broken
  toward fewer parameters (P1 > P2 > P3). Expected fit span ≈ settlements
  2018-11 → 2021-12 (~75 windows) given the daily-history edge + L = 63
  warm-up.
- **Verdict (2022 → 2026, untouched by the fit)**: windows with
  2022-01-01 ≤ s1 ≤ 2026-06-30; the SELECTED candidate only, its own
  eligibility rules. **The verdict statistic is the median ρ_w across
  verdict windows**, compared to 0.20 unrounded. The other two candidates'
  out-of-sample medians are reported as context and CANNOT alter the
  verdict.

## 7. Kill criterion and consequences (pinned; panel language binding)

- **Median out-of-sample correlation < 0.20 → the NULL is recorded**:
  demote the stream permanently — a dated annotation lands in CAPTURES.md
  under `short_volume` barring future proposals from citing daily short
  volume as *positioning* evidence absent venue-complete data. The capture
  itself keeps running (insurance + validation rationale unchanged), and
  the days-to-cover / borrow-screen consumers (panel A4) are explicitly
  unaffected — they consume the bi-monthly SI print, not daily flow.
- **≥ 0.20 → "eligible as timing refinement" ONLY**, behind its own future
  registration (which must then confront the ~T+7-business-day publication
  lag and the venue-coverage caveat head-on). Not a signal, not a ship, no
  SHIP-gate standing.
- Either way this file records the answer permanently; the question is
  closed and may not be relitigated on this data (a venue-complete dataset
  — e.g. exchange-complete short marking — would be a NEW registration).
- Multiple-testing: zero cost to the strategy trial ledger — no returns are
  examined anywhere. The only selection freedom is the disclosed
  3-candidate fit, which never sees 2022+ data; the verdict is computed
  once. No `results/evaluations/` entry is created (this is not a strategy
  trial).

## 8. Pre-registration data QA (disclosed; no hypothesis computation)

Performed before this memo froze, limited to schemas/coverage/flags:
store counts and date ranges; settlement gap histogram (13–19 days,
complete); daily-store completeness vs the NYSE calendar (0 missing days
2018-08+); one-day symbol-overlap and class-share conventions (`BRK/A` ↔
`BRKA`); `split_flag`/`revision_flag` value sets; prev-field vs prior-file
mismatch magnitudes (§5); ADV-field zeros (2,007/20,251 rows on the sampled
partition — FINRA ADV is therefore NOT used anywhere in this design).
No short-volume quantity was ever joined to a short-interest change before
this memo's commit.

## 9. Instrument

`research/a3_nowcast.py` implements exactly this memo; where they could
disagree, the memo wins and the discrepancy is a bug to fix BEFORE the run.
Its tests validate the instrument on synthetic stores only: a planted
flow→ΔSI effect must be detected (ρ ≈ 1), an independent target must read
null, the settlement-lag constants, split exclusion, symbol-join
normalization, coverage floor, and fit/verdict separation are each
asserted. The real stores are touched exactly once, by the registered run.

---
SENTINEL — design frozen above this line. Result appended below by a later
commit; nothing above may change.

## RESULT — 2026-08-01 (the one registered run)

Commit order honored: memo `82b176e` → instrument + synthetic tests
`c916658` → this append. Single invocation of `research/a3_nowcast.py`
(polars 1.38.1, scipy 1.17.1) against the frozen tier exactly as pinned
(2,002 daily files 2017-12-29..2026-07-20; 205 partitions
2017-12-29..2026-06-30). Per-window audit table:
`research/a3_nowcast_windows.json` (184 rows, committed with this append).

**Fit era** — 76 windows, s1 2018-11-15 → 2021-12-31, median 6,019
names/window (skips: 15 windows on the known 2018-01→07 daily CDN hole,
5 on the 63-td baseline warm-up):

| candidate | median fit ρ |
|---|---|
| level | +0.1763 |
| **surprise21 (selected)** | **+0.2305** |
| surprise63 | +0.2237 |

**Verdict era (untouched)** — candidate `surprise21`, 108 windows, s1
2022-01-14 → 2026-06-30, median 7,146 names/window:

- **median OOS Spearman ρ = +0.2414** — IQR [+0.2116, +0.2562],
  **108/108 windows positive**, per-year medians 2022 +0.2129, 2023
  +0.2108, 2024 +0.2492, 2025 +0.2564, 2026 +0.2716.
- Context (non-binding): level +0.1747, surprise63 +0.2301 OOS.
- Short-share echo among eligible names: ~0.44 (fit) / ~0.47 (verdict)
  window medians — consistent with the panel's 57% consolidated-universe
  figure.

**VERDICT per the pinned criterion (§7): +0.2414 ≥ 0.20 → NOT the null.**
The permanent-demotion branch does not fire; no proposal bar is created.
The stream earns exactly and only what §7 grants: **"eligible as timing
refinement", behind its own future registration** — no signal standing, no
SHIP-gate standing, nothing runs on this result. Any future registration
must confront head-on: (a) 22–54% venue coverage, (b) the market-maker
majority of short-marked flow, (c) the ~T+7-business-day publication lag of
the print being nowcast.

**Honest reading, bounded by the printed caveats.** The fit-to-OOS
stability (0.2305 → 0.2414, slightly HIGHER out of sample, monotone-rising
by year) says the trace is real and not an overfit signature — but
ρ ≈ 0.24 means ρ² ≈ 0.06: **~94% of the cross-sectional variance in print
changes is NOT in daily short flow.** The specialist's mechanism critique
stands materially intact — most short-marked volume is liquidity provision;
a minority positioning component exists and survives the venue-coverage
haircut. Both prints are now on the record: the effect is too weak to call
daily short volume "positioning", and too stable to call it nothing.

The question is CLOSED on this data (§7): re-opening requires
venue-complete data under a NEW registration.