# Bounding the Stateful-Purge Leak — Pre-Registration (2026-08-01) **Status: BINDING PRE-REGISTRATION, append-only after today.** Written and committed BEFORE any grid run exists (panel item C4, quant panel report 2026-08-01, finding 10). The runner (`scripts/run_c4_purge_grid.sh`) refuses to execute unless this file is present, and every result JSON records this file's SHA-256. **Production Kelly is untouched** — every override below is runtime-only inside a calibration run; no config file changes. All existing verdicts stand regardless of outcome. --- ## 1. The leak, from first principles CPCV's **purge** deletes a buffer of days around each test block so training cannot *see* the test data through backward-looking features. The buffer is sized to the longest **feature** lookback — momentum's 252 days (`purge_days: 252`, floor-checked against `Strategy.min_required_purge`). But features are not the only channel. The in-sample backtest runs **contiguously through the interior test blocks**, carrying slow *state* across them: - the **Kelly ledger** — per-name holding returns feeding the half-Kelly sizing scalar. Its rolling window is `kelly.rolling_window_months: 24`, implemented as a pool of `months × sleeve_size` observations (`portfolio/construct.py`), i.e. a reach of ≈ **504 trading days** — twice the 252-day purge. A train date up to ~504 days after a test block is sized using test-block returns. - the **kill-switch high-water mark** — equity's running maximum, whose reach is unbounded (any test-block rally raises it forever after). Because the IS metric is computed over train dates whose *sizing* embeds test-block information, this is a leak in the **flattering direction** on the primary kill metric (PBO) — never measured until now. ## 2. Why a 2×2, not a single arm (the red-team redesign) Neither one-knob variant isolates the leak: - Extending the purge alone (252 → 504) removes the Kelly leak **and** deletes ~2× more training data — any PBO move confounds leak-removal with plain design sensitivity to less data. - Shortening the Kelly window alone (24mo → 12mo) removes the leak (reach 252 ≤ purge 252) **and** changes the estimator — any move confounds leak-removal with a different Kelly. The grid crosses both: | cell | `kelly.rolling_window_months` | `purge_days` | Kelly leak present? | |---|---|---|---| | **A** | 24 (production) | 252 (production) | YES (reach 504 > purge 252) | | **B** | 24 | 504 | no (purged away) — **the primary arm** | | C | 12 | 252 | no (reach ≈ 252 ≤ purge) | | D | 12 | 504 | no (both) | The **interaction** — how much bigger the purge effect is when the Kelly window is long enough to leak than when it is not — > `ΔΔPBO = (PBO_A − PBO_B) − (PBO_C − PBO_D)` (and identically for IS-OOS > correlation, `ΔΔcorr`) is the leak: the data-loss sensitivity of a longer purge appears in BOTH rows and cancels; the estimator change appears in BOTH columns and cancels; only the leak lives exclusively in cell A. **The purge-504 arm (B vs A, at production Kelly) is the cleaner single probe and ranks primary in reporting; the DECISION statistic is the interaction only.** Stated limitation, pinned now: the kill-switch high-water-mark channel is window-independent, so it too **cancels in the interaction** — this grid bounds the *Kelly-ledger* leak specifically. The HWM channel remains unmeasured by this design (its reach cannot be windowed); it is recorded here as a known residual, not silently absorbed. ## 3. The pinned materiality bar > The leak is **MATERIAL** iff `|ΔΔPBO| ≥ one path-quantum` **OR** > `|ΔΔcorr| ≥ 0.15`, where one path-quantum = `1 / ceil(n_paths/2)` = > **12.5 pp at the expected 15 paths** — *restated in realized path counts*: > a longer purge shrinks training windows and may drop paths below the > engine's train/test floors, so the summariser recomputes the quantum from > each cell's realized `n_paths` and uses the **largest** per-cell quantum as > the bar. (12.5 pp = one of the 8 top-IS-half paths flipping sides — the > smallest move this statistic can represent at 15 paths.) ## 4. Pre-committed outcomes (all three, before any number exists) 1. **Interaction ≥ bar → STATEFUL-PURGE LEAK, MATERIAL.** The pre-specified fix becomes eligible: extend the purge floor to cover *stateful estimators* — `min_required_purge` (and therefore `cpcv.purge_days`) floored at `max(feature_lookback, kelly.rolling_window_months × 21 td)`. That fix is **one line plus a full re-baseline** (byte-identical guard re-pin + fresh CPCV baselines, per the standing re-baseline rule), shipped as its own follow-up change — never applied silently by this experiment. 2. **A main effect ≥ bar but interaction < bar → DESIGN SENSITIVITY, recorded.** The harness is sensitive to purge length or Kelly window as *design choices* — worth knowing, **not leak evidence**; no purge change. 3. **All cells inside the bar → BOUNDED-IMMATERIAL, recorded.** The leak exists in principle, is now bounded below one path-quantum in practice, and the question closes (cite this memo instead of re-running). In every branch: production `kelly.rolling_window_months` stays 24, no config file changes, all existing verdicts stand. Cell A doubles as a sanity anchor — it should reproduce the standing standard-mode baseline (PBO 75% era); drift there is a harness-integrity flag to investigate before reading the grid. ## 5. The pinned harness As the production standard-mode CPCV, varying nothing but the two grid axes: `run_cpcv`, **n_groups=6, k_test=2, embargo_days=5**, production `MomentumStrategy` (the strategy whose Kelly ledger is the leak vector), full frozen `data/raw` panel, base config `config/settings.yaml` with ONLY the cell's two overrides. purge=504 still satisfies the 252-day feature floor (`test_cpcv_purge_covers_feature_lookback` semantics), so no floor machinery is disturbed. 4 runs ≈ overnight compute. ## 6. Procedure (mechanical) 1. This memo committed BEFORE any run; the runner asserts it exists and stamps its SHA-256 into every result JSON. 2. `scripts/run_c4_purge_grid.sh` runs cells A–D sequentially under `nice`, writing `results/diagnostics/c4_purge_grid/.json` (existing files skipped — resumable). 3. `scripts/summarize_c4_purge_grid.py` computes §3–§4 **mechanically** — the cell table, both interactions, realized-path quanta, and exactly one of the three §4 verdict lines — into `summary.md` beside the JSONs. 4. Any interpretation beyond §4 is appended to THIS memo with a date; the pinned text above is never edited. *Pinned 2026-08-01. Panel-Item: C4.* --- ## VERDICT + FIX APPEND — 2026-08-01 (grid: results/diagnostics/c4_purge_grid/summary.md) Pre-committed outcome 1 fired: **STATEFUL-PURGE LEAK, MATERIAL** — interaction ΔΔcorr = **−0.312** vs the 0.15 bar (ΔΔPBO 0.000 vs 0.125; the corr leg decides, per §3's either/or). Primary arm (B vs A, production Kelly window): purge 252→504 moves PBO 0.750→**0.625** and IS-OOS corr −0.699→**−0.281** with observed Sharpe unchanged (0.780) — the engine is untouched; only the honesty of the split changed. **Fix shipped per §4.1** (same day, its own commit): `Strategy.min_required_purge` now includes the Kelly-ledger reach (`kelly.rolling_window_months × 21 td`) when `risk.sizing: half_kelly`; `cpcv.purge_days` raised 252→**504** (settings.yaml, dated comment); the standing guard test computes and asserts the floor. meanrev (`sizing: none`) untouched at its own keystoned 69. **Re-baseline status:** standard-mode citable numbers = grid cell B (PBO 0.625 / corr −0.281 / obs Sharpe 0.780, n_paths 15). The survivorship-free GOLDEN re-pin was launched and **killed by machine load** (load ~48, second session active) — pending one run of: `thales cpcv --survivorship-free --golden` (now floored at 504). Until it lands, survfree-golden citations carry "pre-C4-floor (PBO 0.500 @ purge 252, 2026-08-01 02:16 run) — re-pin pending". --- ## RE-PIN COMPLETE APPEND — 2026-08-01 evening (closes the pending item above) `thales cpcv --survivorship-free --golden` at the new 504 floor (results/diagnostics/cpcv_2026-08-01T19-08-15.json): | survfree GOLDEN | @ purge 252 (old) | @ purge 504 (new pin) | |---|---|---| | PBO | 0.500 | **0.625** | | IS-OOS corr | −0.42 | **−0.53** | | observed Sharpe | 0.524 | 0.524 (unchanged ✓) | | DSR | 0.665 → 0.611 era | **0.611** @ axis-sets (band 0.490 @ raw) | | OOS paths positive | 15/15 | 15/15 | Note the two modes CONVERGED at PBO 62.5% (standard fell 0.750→0.625, survfree rose 0.500→0.625 — one path-quantum, the statistic's resolution limit). Read through C2's null calibration: 62.5% sits one quantum above the no-skill center (0.500), and corr −0.53 lies inside the no-skill band (null median |corr| 0.443, range −0.02..−0.80) — per the C2 convention it is quoted WITH those percentiles and is not, by itself, evidence of overfitting. The citable honest baseline is now: **survfree GOLDEN @504: PBO 0.625 / corr −0.53 (null-typical) / obs 0.524 / DSR 0.611**.