Your verifier is only as honest as the tool output it reads
Everyone is adding a second model to check the first one. Three papers in Monday's arXiv batch show how easily that checker gets steered by what it is shown.
"Who Verifies the Tools?" started from a production RAG system at the Pilsudski Institute. The tool formatters, the small functions that turn raw results into text for the model, quietly dropped search limits, result caps and identifier namespaces. The agent answered wrong, and the verifier agreed because it saw the same trimmed text. Fixing the formatters flipped one test from verifiers accepting incorrect answers 90 out of 90 times to rejecting them 90 out of 90 times. Same model, same prompt, different tool output. An audit prompt run with Codex and Claude Code found twelve more defects, and pointed at OpenClaw and Hermes it found warped formatters in both. The OpenClaw fix is already merged.
"Trusting the Tool" tested LLM vulnerability auditors against static-analysis evidence across 916 pairs, eleven model configurations from four vendors and 16,572 audits. True evidence barely helped (+0.02). False assurances, a tool saying "this is safe" when it isn't, cut detection of real vulnerabilities by up to 19 points, while false alarms never significantly hurt accuracy. The models defer to good news. A separate per-claim verification call won back 12 points.
Tampered ML, accepted at EMNLP, asked models to judge ML research designs across 1,858 item pairs. Opus 5 scored 77.9%, and the models were sensitive to presentation order and favored flawed designs that "appear more specific, transparent, or conventional."
The shared lesson: a verifier doesn't check reality, it checks a rendering of reality, and whoever controls the rendering controls the verdict. For builders that means auditing your formatters like you audit prompts, never showing the checker the producer's own summary of the evidence, and treating "the tool says it's fine" as the input most likely to fool the judge.
Who Verifies the Tools: https://arxiv.org/abs/2610.32110 Trusting the Tool: https://arxiv.org/abs/2610.32876 Tampered ML: https://arxiv.org/abs/2610.32114
← Back to all articles
"Who Verifies the Tools?" started from a production RAG system at the Pilsudski Institute. The tool formatters, the small functions that turn raw results into text for the model, quietly dropped search limits, result caps and identifier namespaces. The agent answered wrong, and the verifier agreed because it saw the same trimmed text. Fixing the formatters flipped one test from verifiers accepting incorrect answers 90 out of 90 times to rejecting them 90 out of 90 times. Same model, same prompt, different tool output. An audit prompt run with Codex and Claude Code found twelve more defects, and pointed at OpenClaw and Hermes it found warped formatters in both. The OpenClaw fix is already merged.
"Trusting the Tool" tested LLM vulnerability auditors against static-analysis evidence across 916 pairs, eleven model configurations from four vendors and 16,572 audits. True evidence barely helped (+0.02). False assurances, a tool saying "this is safe" when it isn't, cut detection of real vulnerabilities by up to 19 points, while false alarms never significantly hurt accuracy. The models defer to good news. A separate per-claim verification call won back 12 points.
Tampered ML, accepted at EMNLP, asked models to judge ML research designs across 1,858 item pairs. Opus 5 scored 77.9%, and the models were sensitive to presentation order and favored flawed designs that "appear more specific, transparent, or conventional."
The shared lesson: a verifier doesn't check reality, it checks a rendering of reality, and whoever controls the rendering controls the verdict. For builders that means auditing your formatters like you audit prompts, never showing the checker the producer's own summary of the evidence, and treating "the tool says it's fine" as the input most likely to fool the judge.
Who Verifies the Tools: https://arxiv.org/abs/2610.32110 Trusting the Tool: https://arxiv.org/abs/2610.32876 Tampered ML: https://arxiv.org/abs/2610.32114
Comments