Thales
← research journal

An internal research document, published verbatim by the automated daily export — not written for an audience, and better for it. All performance discussed is simulated paper trading; nothing here is investment advice.

DISMISSED — OQP-1 — the decay-verdict ledger records no in-sample provenance, so a verdict scored against the MUTABLE reference curve is permanently indistinguishable from one scored against the pin; dismissed because AUDIT.md's 12-01 register row already pins the fail-closed half verbatim

status: dismissed at adversarial review, panel run 46 (2026-10-28) · raised by the queue-and-clocks lens this run, at the explicit invitation of ops/HANDS.md's 2026-10-26 item ("file the fail-closed half … as a queue row") · class considered: registration hygiene (G5) · judgement considered: the mechanism is REAL and verified; the row is a duplicate · effort of the remaining fragment ~0.05 pd


Plain-language summary

The system has a monthly judge that asks "has the live strategy decayed?" It compares the live equity curve against a frozen in-sample reference. That reference is supposed to be a file pinned once, before the live data existed — results/oos_reference_equity.parquet. If that file is missing, the judge does not stop: it prints a warning and silently uses results/equity_curve.parquet instead, which every backtest overwrites. And the ledger it then writes down — the only evidence feeding the rule that would retire the strategy — records five fields and none of them says which curve it used.

So in principle a verdict could be scored against a curve written minutes earlier, and nobody could ever tell from the ledger.

We are not opening this as a queue item, because the repo already tracks it. AUDIT.md's "before 2026-12-01" register row demands exactly the fail-closed fix, by name, with a deadline and an owner. Filing a queue row would move a record sideways and cost an eviction at the 12-item cap.

Mechanism — verified at source, three times

resolve_reference_path (src/thales/backtest/oos_monitor.py:110-150):

pinned = pinned_reference_path(results_dir)
if Path(pinned).exists():
    return pinned
logger.warning(
    "No pinned OOS reference (%s) — falling back to the MUTABLE %s, which "
    "any backtest run overwrites. ...", pinned, default_curve,
)
return equity_curve_path or default_curve

The pin is gitignored by .gitignore:28 (results/*.parquet) with no un-ignore; git ls-files results/ tracks only oos_verdict_history.jsonl, research_registry.jsonl, lab/ledger.jsonl, baseline_metrics.json and metrics.json. It is also absent from scripts/backup_data.sh's INCLUDE list (data/** only). So the pin exists on one machine, in neither git nor the offsite archive — reproduced from this clean clone, where the fallback is what a run would get.

append_verdict_history (src/thales/execution/go_live.py:97-112) writes a fixed five-key row: {date, verdict, n_live, live_sharpe, live_since}. All five rows in results/diagnostics/oos_verdict_history.jsonl carry exactly those keys and nothing else — no resolved path, no n_is, no is_sharpe. The provenance is printed (cli.py:3056-3074, including a <1000-day warning) but only to transient stdout. That ledger is the sole evidence for the incumbent retire criterion (go_live.py:152-160: two consecutive DEGRADED retires the strategy) and feeds the 2026-12-01 go-live gate.

Why it is dismissed rather than opened

1. The decisive ground: AUDIT.md already pins the fail-closed half, verbatim

AUDIT.md's "before 2026-12-01" register row (raised 2026-10-05, still open 2026-10-19) states the mechanism in full — the gitignore, the missing backup entry, and "oos_monitor.resolve_reference_path does not fail on a missing pin — it warns and scores against the MUTABLE results/equity_curve.parquet" — and then pins:

Pass criteria: the pin is in git (explicit un-ignore, as results/diagnostics/oos_verdict_history.jsonl already is) or in the backup set, AND go-live-gate fails CLOSED rather than scoring on a mutable curve

status ⏳ pending (owner/steward — touches .gitignore, scripts/, and the monitor's failure mode).

That is the row's own declared scope, already registered, already dated, already assigned. Three live records track it: this register row, queue triage 2026-10-26 (which deliberated it by name), and the ops/HANDS.md item. The only genuinely new fragment is three extra keys in a dict literal.

2. The exposure path measured at zero runs in five months

The lens's sharpest claim was a path that manufactures a reference and then persists the verdict: .github/workflows/monthly-revalidation.yml runs thales backtest --start 2010-01-01 (:116), then thales oos-monitor --json-out … (:179), then commits and pushes oos_verdict_history.jsonl to main (:188-196). On a cloud runner the pin cannot exist, so the verdict would be scored against the curve the same job just wrote, and the row would be byte-indistinguishable from a pinned one.

Measured: gh run list --workflow=monthly-revalidation.yml returns exactly one run ever — 2026-06-01, schedule, cancelled at the 120-minute timeout. Zero workflow_dispatch runs in five months. And every ledger commit message is oos: verdict history <date> or a records PR — none carries the workflow's (cloud dispatch …) format, so all five existing rows came from the Mac/interactive path where the pin exists. scripts/monthly_revalidation.sh:88-90 states the design reason in writing: "Runs HERE (local) because the in-sample reference … is gitignored and absent in the cloud runner."

3. "Can you tell afterwards?" — partly yes, outside the ledger

AUDIT.md's 2026-09-08 row records "live Sharpe −1.331 … vs IS +0.795", and 0.795 is the frozen-window pinned baseline. The ledger is silent; the narrative record is not. That weakens "permanently indistinguishable" to "indistinguishable from the ledger alone."

4. On the proposed fail-closed behaviour — it is right, and it is not an outage

Worth recording because it was the one argument against fixing it. Refusing to write a row when the pin is absent does not silence the decay judge: scripts/monthly_revalidation.sh:375-397 already treats a non-zero oos-monitor as a hard failure that emails "🔴 forward-OOS judge FAILED to run" and returns 1 — built after the seven-silent-weeks incident in which a WARN-and-return-0 emailed "✅ re-validation OK". So a refusal is loud. It simply changes nothing today, because the pin exists on the only machine that runs it.

The remaining fragment, and where it belongs

One sentence appended to AUDIT.md's 12-01 pass criteria:

…AND the appended ledger row carries the resolved reference path, n_is and is_sharpe.

Same desk, same deadline, same owner, no eviction, no research_registry.jsonl row, no deflated-Sharpe debt. AUDIT.md is outside the panel's output contract (the panel writes research/queue/ only), so this is a steward or interactive one-liner — recorded here as the carrier so the sentence is not lost.

Reopen conditions — dated and mechanical

Reopen as a row if any becomes true:

  1. The AUDIT.md 12-01 register row is discharged, closed or re-dated WITHOUT the ledger-provenance sentence landing. This dismissal rests entirely on that row carrying the work; if it closes with only the recoverability and fail-closed halves done, the provenance fragment has no home and becomes a row in its own right.
  2. A ledger row is appended from a cloud runner — any oos_verdict_history.jsonl row whose commit is not from the Mac/interactive path (the (cloud dispatch …) message format, or a workflow-authored commit). Mechanical: git log --format='%s' -- results/diagnostics/oos_verdict_history.jsonl. The zero-runs-in-five-months measurement is the whole basis of ground 2 above.
  3. monthly-revalidation.yml's schedule is uncommented, or it is workflow_dispatched for real. Either turns the near-fantasy path into a live one.
  4. A DEGRADED verdict is appended. One DEGRADED puts the retire criterion one row from firing, and at that moment the ledger's inability to say what it was scored against stops being hygiene and becomes the evidence for retiring a strategy.
  5. The December gate is read and oos_monitor_verdict contributes to the decision while the pin is still untracked in git and absent from the backup set.

Explicitly NOT a reopen condition: the passage of time, or a re-verification of the same five-key ledger shape recorded above.

Prior art — checked before proposing

grep -n "oos_reference_equity|resolve_reference_path|pin-oos-reference|pinned reference" across all four research/queue/ directories, TECH_DEBT.md, the frozen research/RESEARCH_QUEUE.md and research/2026-08-01_quant_panel_suggestions.md → zero hits; the slug is genuinely absent from the tree. Distinguished from the nearest rows, each read rather than inferred:

  • open/plb-1 — a different loader (placebo.py load_spy_returns, SPY, the deliberately frozen store). Not this.
  • open/fin-3 — the verdict vocabulary and its power (UNDERPOWERED / MIN_OOS_DAYS), with no reference clause at all (grep "reference" over the file → 0).
  • open/fin-4 — the benchmark being SPY rather than a factor set. Different question.
  • AUDIT.md:"before 2026-12-01" — the ops register, not a queue row, and as established above it is the reason this is a dismissal.