The Receipt Is the Argument
Why every claim we make ships with code you can run.
A number in a slide deck is a claim. A number you can regenerate from the raw data on your own machine is a fact. The distance between those two is the whole discipline.
Most reliability work fails at exactly this seam. The analysis is sound, the chart is clean, and none of it can be rerun. So the reader is left to trust the author. Trust is the thing we are trying to remove from the loop, not add to it.
Our rule is narrow and absolute: every figure we publish reproduces from runnable code, or it does not ship. Not the headline number, not the footnote. The verification script exits non-zero if a single value fails to reproduce, which means the author does not get a pass either. The first time we ran ours, it caught us.
This is what we mean by verified, not asserted. Evals tell you the output is good. A receipt tells you whether that result held, and lets anyone check. Judge where you must. Verify where you can.
Run the autopsy on your own system. agent-reliability is a free, open Claude Code plugin that reproduces the eval-to-production gap, quantifies it by slice, and surfaces well-formed-but-wrong failures, where they exist, that an accuracy score hides. No engagement required.
The tool: github.com/ByteStack-Labs/claude-plugins
The proof: github.com/ByteStack-Labs/agent-reliability-receipts
If it surfaces a silent failure you would rather prove and fix before it ships, that is the work we do. Bring us the receipt and we run the full Production ML Autopsy: reproduce the failure, prove the root cause, and hand you a report where every number reruns. Book a Production ML Autopsy →
Jesse Moses is the Founder & Chief Architect of ByteStack Labs, a production-reliability firm for AI and ML systems. ByteStack Labs offers Diagnostic, Architecture & Engineering, and Advisory engagements at bytestacklabs.com.