Thales
← research journal
Aug 1, 2026raw markdown ↗

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

Bounding the Stateful-Purge Leak — Pre-Registration (2026-08-01)

Status: BINDING PRE-REGISTRATION, append-only after today. Written and committed BEFORE any grid run exists (panel item C4, quant panel report 2026-08-01, finding 10). The runner (scripts/run_c4_purge_grid.sh) refuses to execute unless this file is present, and every result JSON records this file's SHA-256. Production Kelly is untouched — every override below is runtime-only inside a calibration run; no config file changes. All existing verdicts stand regardless of outcome.


1. The leak, from first principles

CPCV's purge deletes a buffer of days around each test block so training cannot see the test data through backward-looking features. The buffer is sized to the longest feature lookback — momentum's 252 days (purge_days: 252, floor-checked against Strategy.min_required_purge).

But features are not the only channel. The in-sample backtest runs contiguously through the interior test blocks, carrying slow state across them:

  • the Kelly ledger — per-name holding returns feeding the half-Kelly sizing scalar. Its rolling window is kelly.rolling_window_months: 24, implemented as a pool of months × sleeve_size observations (portfolio/construct.py), i.e. a reach of ≈ 504 trading days — twice the 252-day purge. A train date up to ~504 days after a test block is sized using test-block returns.
  • the kill-switch high-water mark — equity's running maximum, whose reach is unbounded (any test-block rally raises it forever after).

Because the IS metric is computed over train dates whose sizing embeds test-block information, this is a leak in the flattering direction on the primary kill metric (PBO) — never measured until now.

2. Why a 2×2, not a single arm (the red-team redesign)

Neither one-knob variant isolates the leak:

  • Extending the purge alone (252 → 504) removes the Kelly leak and deletes ~2× more training data — any PBO move confounds leak-removal with plain design sensitivity to less data.
  • Shortening the Kelly window alone (24mo → 12mo) removes the leak (reach 252 ≤ purge 252) and changes the estimator — any move confounds leak-removal with a different Kelly.

The grid crosses both:

cellkelly.rolling_window_monthspurge_daysKelly leak present?
A24 (production)252 (production)YES (reach 504 > purge 252)
B24504no (purged away) — the primary arm
C12252no (reach ≈ 252 ≤ purge)
D12504no (both)

The interaction — how much bigger the purge effect is when the Kelly window is long enough to leak than when it is not —

ΔΔPBO = (PBO_A − PBO_B) − (PBO_C − PBO_D) (and identically for IS-OOS correlation, ΔΔcorr)

is the leak: the data-loss sensitivity of a longer purge appears in BOTH rows and cancels; the estimator change appears in BOTH columns and cancels; only the leak lives exclusively in cell A. The purge-504 arm (B vs A, at production Kelly) is the cleaner single probe and ranks primary in reporting; the DECISION statistic is the interaction only.

Stated limitation, pinned now: the kill-switch high-water-mark channel is window-independent, so it too cancels in the interaction — this grid bounds the Kelly-ledger leak specifically. The HWM channel remains unmeasured by this design (its reach cannot be windowed); it is recorded here as a known residual, not silently absorbed.

3. The pinned materiality bar

The leak is MATERIAL iff |ΔΔPBO| ≥ one path-quantum OR |ΔΔcorr| ≥ 0.15, where one path-quantum = 1 / ceil(n_paths/2) = 12.5 pp at the expected 15 pathsrestated in realized path counts: a longer purge shrinks training windows and may drop paths below the engine's train/test floors, so the summariser recomputes the quantum from each cell's realized n_paths and uses the largest per-cell quantum as the bar. (12.5 pp = one of the 8 top-IS-half paths flipping sides — the smallest move this statistic can represent at 15 paths.)

4. Pre-committed outcomes (all three, before any number exists)

  1. Interaction ≥ bar → STATEFUL-PURGE LEAK, MATERIAL. The pre-specified fix becomes eligible: extend the purge floor to cover stateful estimatorsmin_required_purge (and therefore cpcv.purge_days) floored at max(feature_lookback, kelly.rolling_window_months × 21 td). That fix is one line plus a full re-baseline (byte-identical guard re-pin + fresh CPCV baselines, per the standing re-baseline rule), shipped as its own follow-up change — never applied silently by this experiment.
  2. A main effect ≥ bar but interaction < bar → DESIGN SENSITIVITY, recorded. The harness is sensitive to purge length or Kelly window as design choices — worth knowing, not leak evidence; no purge change.
  3. All cells inside the bar → BOUNDED-IMMATERIAL, recorded. The leak exists in principle, is now bounded below one path-quantum in practice, and the question closes (cite this memo instead of re-running).

In every branch: production kelly.rolling_window_months stays 24, no config file changes, all existing verdicts stand. Cell A doubles as a sanity anchor — it should reproduce the standing standard-mode baseline (PBO 75% era); drift there is a harness-integrity flag to investigate before reading the grid.

5. The pinned harness

As the production standard-mode CPCV, varying nothing but the two grid axes: run_cpcv, n_groups=6, k_test=2, embargo_days=5, production MomentumStrategy (the strategy whose Kelly ledger is the leak vector), full frozen data/raw panel, base config config/settings.yaml with ONLY the cell's two overrides. purge=504 still satisfies the 252-day feature floor (test_cpcv_purge_covers_feature_lookback semantics), so no floor machinery is disturbed. 4 runs ≈ overnight compute.

6. Procedure (mechanical)

  1. This memo committed BEFORE any run; the runner asserts it exists and stamps its SHA-256 into every result JSON.
  2. scripts/run_c4_purge_grid.sh runs cells A–D sequentially under nice, writing results/diagnostics/c4_purge_grid/<cell>.json (existing files skipped — resumable).
  3. scripts/summarize_c4_purge_grid.py computes §3–§4 mechanically — the cell table, both interactions, realized-path quanta, and exactly one of the three §4 verdict lines — into summary.md beside the JSONs.
  4. Any interpretation beyond §4 is appended to THIS memo with a date; the pinned text above is never edited.

Pinned 2026-08-01. Panel-Item: C4.


VERDICT + FIX APPEND — 2026-08-01 (grid: results/diagnostics/c4_purge_grid/summary.md)

Pre-committed outcome 1 fired: STATEFUL-PURGE LEAK, MATERIAL — interaction ΔΔcorr = −0.312 vs the 0.15 bar (ΔΔPBO 0.000 vs 0.125; the corr leg decides, per §3's either/or). Primary arm (B vs A, production Kelly window): purge 252→504 moves PBO 0.750→0.625 and IS-OOS corr −0.699→−0.281 with observed Sharpe unchanged (0.780) — the engine is untouched; only the honesty of the split changed.

Fix shipped per §4.1 (same day, its own commit): Strategy.min_required_purge now includes the Kelly-ledger reach (kelly.rolling_window_months × 21 td) when risk.sizing: half_kelly; cpcv.purge_days raised 252→504 (settings.yaml, dated comment); the standing guard test computes and asserts the floor. meanrev (sizing: none) untouched at its own keystoned 69.

Re-baseline status: standard-mode citable numbers = grid cell B (PBO 0.625 / corr −0.281 / obs Sharpe 0.780, n_paths 15). The survivorship-free GOLDEN re-pin was launched and killed by machine load (load ~48, second session active) — pending one run of: thales cpcv --survivorship-free --golden (now floored at 504). Until it lands, survfree-golden citations carry "pre-C4-floor (PBO 0.500 @ purge 252, 2026-08-01 02:16 run) — re-pin pending".


RE-PIN COMPLETE APPEND — 2026-08-01 evening (closes the pending item above)

thales cpcv --survivorship-free --golden at the new 504 floor (results/diagnostics/cpcv_2026-08-01T19-08-15.json):

survfree GOLDEN@ purge 252 (old)@ purge 504 (new pin)
PBO0.5000.625
IS-OOS corr−0.42−0.53
observed Sharpe0.5240.524 (unchanged ✓)
DSR0.665 → 0.611 era0.611 @ axis-sets (band 0.490 @ raw)
OOS paths positive15/1515/15

Note the two modes CONVERGED at PBO 62.5% (standard fell 0.750→0.625, survfree rose 0.500→0.625 — one path-quantum, the statistic's resolution limit). Read through C2's null calibration: 62.5% sits one quantum above the no-skill center (0.500), and corr −0.53 lies inside the no-skill band (null median |corr| 0.443, range −0.02..−0.80) — per the C2 convention it is quoted WITH those percentiles and is not, by itself, evidence of overfitting. The citable honest baseline is now: survfree GOLDEN @504: PBO 0.625 / corr −0.53 (null-typical) / obs 0.524 / DSR 0.611.