DISMISSED — RCX-1 — portfolio reconcile reports local corruption as "broker unreachable" and exits 0, and the comment saying that was fixed describes a mechanism that cannot work
status: dismissed at adversarial review, panel run 25 (2026-09-17) · class considered: inert guard / loudness (G2) · surfaced by the change-review lens while verifying PR #171's own cost argument, so it is outside the delta — the lines are unchanged since long before this run · the defect is real and verified; it is dismissed on reachability and slot cost, with the comment fix handed over
What was considered, and what is true
thales portfolio reconcile — the designated book-integrity detector, run
if: always() after the trade step and the place RUNBOOK's "when a run goes
red" sends the operator — swallows local state corruption, mislabels it as
a broker problem, and exits 0 (a green step).
Verified at HEAD dfde575:
src/thales/cli.py:4205openstry:;:4215-4218carries the comment "Read local state FIRST so a corrupt/torn local JSONL raises OUT of the broker-only except below (adversarial review 2026-07-18: the blanket except mislabeled local corruption as 'broker unreachable' and exited 0 — a GREEN reconcile on a corrupt book)";:4219-4220are the two local reads — inside that same try;:4222isexcept Exception as e:, unconditional;:4226-4227print "reconcile skipped — broker unreachable" andraise typer.Exit(code=0);src/thales/execution/state.py:64-74—_read_jsonlcalls barejson.loads, so a torn line raisesJSONDecodeError⊂ValueError⊂Exception.
Moving a statement earlier within the same try cannot change which handler
catches it, so the stated remedy is a no-op against its stated failure mode.
The correct shape exists 50 lines away in the same file: backfill-equity
(:4272-4280) wraps only the broker call, with the local read outside it.
No test pins any of this (grep -rn "broker unreachable" tests/ returns only
test_go_live.py's safety-halt strings).
Two refinements found during review:
- The message is wrong in both directions.
_make_broker(...)is outside the try, so a genuine credential/construction failure is never caught by this handler at all; it only ever covers failures insidereconcile(). - The 2026-07-18 attribution is NOT-VERIFIED. This checkout is shallow
(
git log -L 4215,4220bottoms out at 2026-09-09), so what that review actually changed cannot be read here. The "a fix is on the record as having closed a hole it did not close" argument rests on the comment's own text, not on the reviewed diff — and is recorded that way deliberately.
Why it was dismissed
Reachability is engineered away at both writers, and that was measured, not
assumed. state.py:47-61 _append_jsonl flushes + os.fsyncs and argues a
single JSON line under PIPE_BUF (~4 KiB) is written atomically by POSIX;
atomic_write_lines (:31-45) uses temp-file + os.replace. Measured on the
real committed logs: data/processed/order_log.jsonl 544 lines / 650 B
max, data/processed/equity_history.jsonl 123 / 86 B,
data/state/meanrev/order_log.jsonl 3,984 / 403 B,
data/state/vrp/order_log.jsonl 55 / 325 B — every record ~6× under the
limit.
That leaves one live path: a hand-edit of a committed state file, which is
not hypothetical (PR #125, 2026-08-31, corrected the 08-28 phantom equity mark
by deleting a line by hand and re-running backfill-equity). But in that
scenario the trading run reads the same state early and reds loudly, so
reconcile's mislabel costs diagnosis time, not detection.
A one-line typed except is not worth a slot against a 12/12 cap. The
dismissal is free and keeps the tripwire.
Handed over instead
Fix the comment even if the except is never touched. A comment that
credits a review with closing a hole it did not close is the part that spends
trust, and it costs nothing to correct. Suggested for TECH_DEBT.md so it
gets done in passing rather than waiting on a queue slot: narrow the try to the
broker call (mirroring backfill-equity 50 lines below), or type the except —
and either way delete or correct the claim at cli.py:4215-4218.
Reopens if
- (a) any committed state JSONL line exceeds 4096 B — one
awkoverdata/**/*.jsonl, cheap enough for triage or capture-QA to run — because that is the moment the writer-side atomicity argument stops holding and torn lines become reachable; or - (b) a local read failure is ever observed reported as "broker unreachable" in a run log — the true confirmation; or
- (c) this over-broad-
except-around-a-broker-call shape is found at a second site. The shop's own record says this class recurs (IGD-1, AMG-4, PR #168's dark-routine branch, DGX-1 leg 3), and a second instance is what would justify a structural control — a test asserting that everyexcept Exceptionwrapping a broker call names its exception types — rather than a one-liner.
Deliberately NOT a reopen condition: "the owner hand-edits a state file again." No instrument can observe it, and it asks him to remember a queue row at the exact moment he is doing something risky. The comment fix above is the mitigation for that path.
Prior art checked
No rcx-* slug anywhere in the queue tree or the frozen
research/RESEARCH_QUEUE.md. Not covered by IGD-1 (built — five discarded
bool returns, a different shape), OSR-1 (open — broker terminal-status
blindness), EXQ-1 (built — batch-quote fail-open), or PIT-1 (open —
the nearest class, not this instance).
Panel run 45 append (2026-10-27) — reopen condition (c) has FIRED, over-satisfied ten times over, and the second site makes yesterday's brand-new not_found / lookup_failed split unreachable
Recorded, not reopened: dismissed/ → open/ is the steward's move
(ops/STEWARD.md:269-270 — "Reopen a dismissed item only when its own
written reopen condition has fired — cite it"). This append is the citation
that move needs. Precedent for a dated append rather than a mint:
rdr-1's own 09-25 annotation on its fired leg (b), plus b4-notch,
bcn-1 and lck-1.
The condition, verbatim from :88-94 above: "(c) this
over-broad-except-around-a-broker-call shape is found at a second site.
The shop's own record says this class recurs (IGD-1, AMG-4, PR #168's
dark-routine branch, DGX-1 leg 3), and a second instance is what would justify
a structural control — a test asserting that every except Exception wrapping
a broker call names its exception types — rather than a one-liner."
The second site, verified at HEAD 781343e:
src/thales/execution/alpaca_broker.py:428-432
def get_order_by_client_id(self, client_order_id: str) -> Order | None:
try:
result = self.client.get_order_by_client_id(client_order_id)
except Exception:
return None
It is not one second site. grep -c 'except Exception' src/thales/execution/alpaca_broker.py → 10. Condition (c) asked for a
second instance to justify a structural control; the file carries ten, so the
control is the proportionate remedy and the one-liner is not.
Found by the change-review lens while reviewing #285, which touched this exact
function (for the multi-leg symbol=None crash) and correctly left the handler
alone — the TypeError it fixed was raised below the try.
The consequence: a transient broker failure is reported as an unresolvable book-vs-broker divergence, with order ids named
status_sweep.py:180-186 files aNonereturn asrep.not_foundunder the comment "NOT a lag. We recorded this order as submitted and the broker cannot find it under the id we minted, so one of the two records is wrong and no future sweep will resolve it either."cli.py:4366-4374 (thales portfolio sweep-status) prints that count with "these cannot be resolved by a later sweep" and names up to 20client_order_ids;cli.py:1119-1122 (thales run) prints the same summary and names up to 10.thales runsweeps after trading every weekday on tbx-win, three sleeves.
So a 429 after alpaca-py's own retries exhaust, a read timeout from the
deliberately injected 30 s deadline at alpaca_broker.py:54-67, a 5xx, or an
expired key reaches the operator as a permanent divergence. The truth is the
opposite: the next sweep would have resolved it. And the real divergence class
— what OSR-1 was built for — is diluted by transients in the same bucket.
Measured through the real AlpacaBroker, twice independently (change-review
lens, then the adversarial reviewer):
| injected at the SDK client | sweep reports |
|---|---|
APIError 429 rate limit | not found, not lookup error + the id named |
| read timeout | not found |
APIError 401 × 3 | 3 not found, 0 lookup error(s) + three ids named |
| a broker object that raises | the only way lookup_failed fires |
APIError("order not found") vs APIError("500 internal server error") | byte-identical reports |
Grounding suites green on the unmodified tree:
tests/test_execution/{test_alpaca_broker,test_failed_streaks,test_status_sweep}.py
→ 75 passed;
{test_digest,test_fleet_digest,test_pipeline,test_vrp_fidelity,test_state}.py
→ 115 passed.
lookup_failed is NEWLY unreachable — it worked for one day, by accident
Stated precisely, because "structurally dead" would be wrong. #285's PR body:
"8/8 VRP orders since 09-04 resolve to FILLED. Before the fix: 0, with 8
errors." Pre-#285 the TypeError was raised at the result.symbol use,
outside the try that wraps only the client call, so it propagated to the
sweep's own handler — lookup_failed was the one bucket VRP ever reached. #285
fixed that bug and in doing so removed the sole production path into the
bucket: AlpacaBroker now never raises, and SimulatedBroker.get_order_by_client_id
(simulated.py:107-108) is a dict.get. The split did not fail to work. It
worked for a day and then sealed itself, the day after the reporting it feeds
was merged (#270, 2026-10-26).
Why this matters beyond the mislabel: a memo pinned a claim on this instrument
research/2026-10-22_a_sweep_that_resolves_nothing_says_nothing.md names two
indistinguishable causes for the VRP silence — "Alpaca does not resolve a
multi-leg parent … so the lookup returned None" vs "the lookup raised and
was swallowed" — and says the fix is "deliberately the one that makes the
next run answer the question by itself." It cannot: both causes land in
not_found. The question was in fact answered by #285's credentialed dry
run, not by the instrument built to answer it.
And the gate that waits on that answer is still worded as if the instrument
will supply it: fleet_digest.py:1553-1557, inside digest_exceptions
(def at :1465 — not _gather_option_sleeve, which is DGX-1 leg 1's
subject and a different defect), reads "…which is what will answer the lookup
question. Wire this line only after it is answered." The precondition
cleared on 2026-10-26 by a credentialed run nine seconds before that comment
merged. Recorded here rather than on DGX-1 because the fact belongs to the
instrument, not to the option gather's missing key.
Discharge criterion — rewritten at adversarial review, because (c)'s own wording has a wriggle
"Names its exception types" could be satisfied by narrowing :432 alone while
nine except Exception remain, or by naming a type in a comment. So:
Discharged only when a test enumerates every
exceptclause insrc/thales/execution/alpaca_broker.pywhose body wraps or follows a broker call and asserts each names concrete exception types (not bareException), andthales portfolio sweep-statusreports a transient broker failure inlookup_failedand never innot_found, proven by a test that injects a 429 through the realAlpacaBrokerand asserts the bucket. Not discharged by fixing:432alone: the count at filing is 10except Exceptionin that one file, and the test must fail if a new one is added.
The "no future sweep will resolve it either" comment at status_sweep.py:182-184
must be deleted or qualified in the same change, or the code keeps asserting
something false.
Remedy and cost, unchanged from what this file already specified: type the
handler (or return a distinct sentinel / re-raise a typed error) so a transient
lands in lookup_failed and a genuine absence in not_found — ~0.15 pd — plus
(c)'s own structural control, ~0.1 pd. Observability tier; no trading-path
behaviour change; sweep_terminal_statuses keeps its signature and return.