Bounding the Stateful-Purge Leak — Pre-Registration (2026-08-01)
Status: BINDING PRE-REGISTRATION, append-only after today. Written and
committed BEFORE any grid run exists (panel item C4, quant panel report
2026-08-01, finding 10). The runner (scripts/run_c4_purge_grid.sh) refuses
to execute unless this file is present, and every result JSON records this
file's SHA-256. Production Kelly is untouched — every override below is
runtime-only inside a calibration run; no config file changes. All existing
verdicts stand regardless of outcome.
1. The leak, from first principles
CPCV's purge deletes a buffer of days around each test block so training
cannot see the test data through backward-looking features. The buffer is
sized to the longest feature lookback — momentum's 252 days
(purge_days: 252, floor-checked against Strategy.min_required_purge).
But features are not the only channel. The in-sample backtest runs contiguously through the interior test blocks, carrying slow state across them:
- the Kelly ledger — per-name holding returns feeding the half-Kelly
sizing scalar. Its rolling window is
kelly.rolling_window_months: 24, implemented as a pool ofmonths × sleeve_sizeobservations (portfolio/construct.py), i.e. a reach of ≈ 504 trading days — twice the 252-day purge. A train date up to ~504 days after a test block is sized using test-block returns. - the kill-switch high-water mark — equity's running maximum, whose reach is unbounded (any test-block rally raises it forever after).
Because the IS metric is computed over train dates whose sizing embeds test-block information, this is a leak in the flattering direction on the primary kill metric (PBO) — never measured until now.
2. Why a 2×2, not a single arm (the red-team redesign)
Neither one-knob variant isolates the leak:
- Extending the purge alone (252 → 504) removes the Kelly leak and deletes ~2× more training data — any PBO move confounds leak-removal with plain design sensitivity to less data.
- Shortening the Kelly window alone (24mo → 12mo) removes the leak (reach 252 ≤ purge 252) and changes the estimator — any move confounds leak-removal with a different Kelly.
The grid crosses both:
| cell | kelly.rolling_window_months | purge_days | Kelly leak present? |
|---|---|---|---|
| A | 24 (production) | 252 (production) | YES (reach 504 > purge 252) |
| B | 24 | 504 | no (purged away) — the primary arm |
| C | 12 | 252 | no (reach ≈ 252 ≤ purge) |
| D | 12 | 504 | no (both) |
The interaction — how much bigger the purge effect is when the Kelly window is long enough to leak than when it is not —
ΔΔPBO = (PBO_A − PBO_B) − (PBO_C − PBO_D)(and identically for IS-OOS correlation,ΔΔcorr)
is the leak: the data-loss sensitivity of a longer purge appears in BOTH rows and cancels; the estimator change appears in BOTH columns and cancels; only the leak lives exclusively in cell A. The purge-504 arm (B vs A, at production Kelly) is the cleaner single probe and ranks primary in reporting; the DECISION statistic is the interaction only.
Stated limitation, pinned now: the kill-switch high-water-mark channel is window-independent, so it too cancels in the interaction — this grid bounds the Kelly-ledger leak specifically. The HWM channel remains unmeasured by this design (its reach cannot be windowed); it is recorded here as a known residual, not silently absorbed.
3. The pinned materiality bar
The leak is MATERIAL iff
|ΔΔPBO| ≥ one path-quantumOR|ΔΔcorr| ≥ 0.15, where one path-quantum =1 / ceil(n_paths/2)= 12.5 pp at the expected 15 paths — restated in realized path counts: a longer purge shrinks training windows and may drop paths below the engine's train/test floors, so the summariser recomputes the quantum from each cell's realizedn_pathsand uses the largest per-cell quantum as the bar. (12.5 pp = one of the 8 top-IS-half paths flipping sides — the smallest move this statistic can represent at 15 paths.)
4. Pre-committed outcomes (all three, before any number exists)
- Interaction ≥ bar → STATEFUL-PURGE LEAK, MATERIAL. The pre-specified
fix becomes eligible: extend the purge floor to cover stateful
estimators —
min_required_purge(and thereforecpcv.purge_days) floored atmax(feature_lookback, kelly.rolling_window_months × 21 td). That fix is one line plus a full re-baseline (byte-identical guard re-pin + fresh CPCV baselines, per the standing re-baseline rule), shipped as its own follow-up change — never applied silently by this experiment. - A main effect ≥ bar but interaction < bar → DESIGN SENSITIVITY, recorded. The harness is sensitive to purge length or Kelly window as design choices — worth knowing, not leak evidence; no purge change.
- All cells inside the bar → BOUNDED-IMMATERIAL, recorded. The leak exists in principle, is now bounded below one path-quantum in practice, and the question closes (cite this memo instead of re-running).
In every branch: production kelly.rolling_window_months stays 24, no config
file changes, all existing verdicts stand. Cell A doubles as a sanity anchor —
it should reproduce the standing standard-mode baseline (PBO 75% era); drift
there is a harness-integrity flag to investigate before reading the grid.
5. The pinned harness
As the production standard-mode CPCV, varying nothing but the two grid axes:
run_cpcv, n_groups=6, k_test=2, embargo_days=5, production
MomentumStrategy (the strategy whose Kelly ledger is the leak vector), full
frozen data/raw panel, base config config/settings.yaml with ONLY the
cell's two overrides. purge=504 still satisfies the 252-day feature floor
(test_cpcv_purge_covers_feature_lookback semantics), so no floor machinery
is disturbed. 4 runs ≈ overnight compute.
6. Procedure (mechanical)
- This memo committed BEFORE any run; the runner asserts it exists and stamps its SHA-256 into every result JSON.
scripts/run_c4_purge_grid.shruns cells A–D sequentially undernice, writingresults/diagnostics/c4_purge_grid/<cell>.json(existing files skipped — resumable).scripts/summarize_c4_purge_grid.pycomputes §3–§4 mechanically — the cell table, both interactions, realized-path quanta, and exactly one of the three §4 verdict lines — intosummary.mdbeside the JSONs.- Any interpretation beyond §4 is appended to THIS memo with a date; the pinned text above is never edited.
Pinned 2026-08-01. Panel-Item: C4.
VERDICT + FIX APPEND — 2026-08-01 (grid: results/diagnostics/c4_purge_grid/summary.md)
Pre-committed outcome 1 fired: STATEFUL-PURGE LEAK, MATERIAL — interaction ΔΔcorr = −0.312 vs the 0.15 bar (ΔΔPBO 0.000 vs 0.125; the corr leg decides, per §3's either/or). Primary arm (B vs A, production Kelly window): purge 252→504 moves PBO 0.750→0.625 and IS-OOS corr −0.699→−0.281 with observed Sharpe unchanged (0.780) — the engine is untouched; only the honesty of the split changed.
Fix shipped per §4.1 (same day, its own commit):
Strategy.min_required_purge now includes the Kelly-ledger reach
(kelly.rolling_window_months × 21 td) when risk.sizing: half_kelly;
cpcv.purge_days raised 252→504 (settings.yaml, dated comment); the
standing guard test computes and asserts the floor. meanrev
(sizing: none) untouched at its own keystoned 69.
Re-baseline status: standard-mode citable numbers = grid cell B
(PBO 0.625 / corr −0.281 / obs Sharpe 0.780, n_paths 15). The
survivorship-free GOLDEN re-pin was launched and killed by machine load
(load ~48, second session active) — pending one run of:
thales cpcv --survivorship-free --golden (now floored at 504). Until it
lands, survfree-golden citations carry "pre-C4-floor (PBO 0.500 @ purge
252, 2026-08-01 02:16 run) — re-pin pending".
RE-PIN COMPLETE APPEND — 2026-08-01 evening (closes the pending item above)
thales cpcv --survivorship-free --golden at the new 504 floor
(results/diagnostics/cpcv_2026-08-01T19-08-15.json):
| survfree GOLDEN | @ purge 252 (old) | @ purge 504 (new pin) |
|---|---|---|
| PBO | 0.500 | 0.625 |
| IS-OOS corr | −0.42 | −0.53 |
| observed Sharpe | 0.524 | 0.524 (unchanged ✓) |
| DSR | 0.665 → 0.611 era | 0.611 @ axis-sets (band 0.490 @ raw) |
| OOS paths positive | 15/15 | 15/15 |
Note the two modes CONVERGED at PBO 62.5% (standard fell 0.750→0.625, survfree rose 0.500→0.625 — one path-quantum, the statistic's resolution limit). Read through C2's null calibration: 62.5% sits one quantum above the no-skill center (0.500), and corr −0.53 lies inside the no-skill band (null median |corr| 0.443, range −0.02..−0.80) — per the C2 convention it is quoted WITH those percentiles and is not, by itself, evidence of overfitting. The citable honest baseline is now: survfree GOLDEN @504: PBO 0.625 / corr −0.53 (null-typical) / obs 0.524 / DSR 0.611.