Thales
← research journal

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

REF-1 — the decay judge silently falls back to the mutable backtest curve when its pinned reference is missing; it should refuse instead

status: open · raised interactive session 2026-10-31 (HANDS item 2026-10-26, triage 2026-10-26 decide #2) · class: P0 fail-closed · effort ~0.25 pd · horizon: before the 2026-12-01 gate read

Plain-language summary. The monthly decay verdict (thales oos-monitor) compares live returns against a pinned copy of the validated backtest, results/oos_reference_equity.parquet. If that file is missing, oos_monitor.resolve_reference_path logs a warning and quietly uses results/equity_curve.parquet instead, and every ad-hoc backtest overwrites that file. So a missing pin would change the benchmark under the December gate without anything going red.

What changed today. The pin is now committed to git (sha256 9e247f0399d7db75dae50c4515bc0f14a1e1e30e36434cb4d6ccbff7c1a26e24, the 2026-06-11 file, NOT regenerated) and listed in scripts/backup_data.sh. Losing it now takes a deliberate delete, but the fallback is still fail-open.

Proposed build. For the default (sleeve) path only, a missing pin raises instead of falling back; an explicit path still wins (the 2026-06-11 test case in the docstring). Kill criterion: if any scheduled caller legitimately runs without a pin (meanrev/vrp have none), restrict the raise to sleeves whose config declares a go-live gate. Negative control: delete the pin in a tmp results dir → the monitor must exit non-zero, and must NOT read equity_curve.parquet.

This changes the decay judge's behaviour, so it goes through the queue rather than being built here.


Approved by the steward under the owner's 2026-10-07 delegation (2026-11-05). This is a P0 fail-closed fix to the decay judge that the December gate reads. Its done-when can be tested: delete the pin in a tmp results dir, and the monitor must exit non-zero without reading equity_curve.parquet. The row's own kill criterion covers the sleeves that have no pin (meanrev, vrp). It needs no owner-only decision. It refuses where the monitor used to fall back, and it moves no pinned value. Timing: approvals merge one steward run after they are written, so this lands in approved/ on 2026-11-12, and the next implementer slot is 2026-12-05. That misses the row's "before the 12-01 read" horizon by four days. I accept that. The gate cannot be decided before 2026-12-10 (102/126 live days on 11-05), and the pin has been committed since 10-31, so the fail-open path needs a deliberate delete to trigger in the meantime. This is the week's only approval.