This dispatch is built around one test:
“If this wanted to lie to me, could it? If yes, it doesn’t ship.”
It starts from an audit of an agent system whose dashboard showed green while its test suite was mostly failing. From there it works out what it takes to build agent systems that tell the truth: systems honest enough to fail loudly, instead of reporting success they have not earned.
The audit, with its numbers, is told on the home page.
The full piece is on FutureLab: The Anti-Theater Principle: Building Agent Systems That Tell the Truth (opens in a new tab).
