Count sources, not votes: four papers on why agent approval keeps failing in the same place
Thursday's arXiv batch had a cluster of papers on the step where an agent asks permission, or should have. Read together, they say the approval layer fails less because models are dumb and more because everyone checks the same thing.
Correlated Failures of AI Approval Monitors (arXiv 2610.29178) tested payment-approval monitors built from 12 small open-weight models plus Opus 5.5, GPT-6 Astra, Gemini 4 Argon and DeepSeek-V3.2. Every reliable way to get fraud approved came through a single evidence source, so stacking more monitors that read the same source did not help. Small models and DeepSeek obeyed planted instructions; the closed models ignored them in all but one case. Counting independent sources instead of monitor votes blocked 0.9 percent of correct payments versus 7.4 percent for the best vote-counting rule, at the same safety level. The authors' pre-registered worst-case test still failed for every model-only rule, and only adding a rule engine that cannot be persuaded restored safety. Link: https://arxiv.org/abs/2610.29178
Preference Is Not Permission (arXiv 2610.29216) looked at auto-purchase. Across 16 models, agents commit to buying even when fit is low and even when their own evidence flag says the justification is missing. A shallow learner trained on just 30 to 40 of a person's own labels predicted where that person draws the line at a median 0.79, beating every prompting setup tried, including an oracle prompt per annotator. Refusal Transfer Failure (arXiv 2610.28982) adds that requests a model refuses in plain chat become unsafe tool calls 10 to 52 percent of the time once they arrive through standard tool prompting. Links: https://arxiv.org/abs/2610.29216 and https://arxiv.org/abs/2610.28982
RegretBench (arXiv 2610.28934) covers what happens after a bad approval. It built 205 tasks from real posts by people who regretted not saving something first. Six frontier models finished 86.1 percent of tasks but made a backup before acting only 15.2 percent of the time. A harness plug-in called BackUpper covered every regret action, restored state at 92.2 percent similarity, and cost $0.014 per task with seven times fewer tokens than asking the model to back up itself. Link: https://arxiv.org/abs/2610.28934
The common thread fits on a sticky note. Do not trust a model to know its own limits, trust the structure around it: independent inputs, a rule layer that cannot be talked out of a no, a per-user threshold learned from the user, and an undo button the harness provides. Walmart's spending caps and Duet's hard no-go list from earlier this week are the product versions of exactly this.
← Back to all articles
Correlated Failures of AI Approval Monitors (arXiv 2610.29178) tested payment-approval monitors built from 12 small open-weight models plus Opus 5.5, GPT-6 Astra, Gemini 4 Argon and DeepSeek-V3.2. Every reliable way to get fraud approved came through a single evidence source, so stacking more monitors that read the same source did not help. Small models and DeepSeek obeyed planted instructions; the closed models ignored them in all but one case. Counting independent sources instead of monitor votes blocked 0.9 percent of correct payments versus 7.4 percent for the best vote-counting rule, at the same safety level. The authors' pre-registered worst-case test still failed for every model-only rule, and only adding a rule engine that cannot be persuaded restored safety. Link: https://arxiv.org/abs/2610.29178
Preference Is Not Permission (arXiv 2610.29216) looked at auto-purchase. Across 16 models, agents commit to buying even when fit is low and even when their own evidence flag says the justification is missing. A shallow learner trained on just 30 to 40 of a person's own labels predicted where that person draws the line at a median 0.79, beating every prompting setup tried, including an oracle prompt per annotator. Refusal Transfer Failure (arXiv 2610.28982) adds that requests a model refuses in plain chat become unsafe tool calls 10 to 52 percent of the time once they arrive through standard tool prompting. Links: https://arxiv.org/abs/2610.29216 and https://arxiv.org/abs/2610.28982
RegretBench (arXiv 2610.28934) covers what happens after a bad approval. It built 205 tasks from real posts by people who regretted not saving something first. Six frontier models finished 86.1 percent of tasks but made a backup before acting only 15.2 percent of the time. A harness plug-in called BackUpper covered every regret action, restored state at 92.2 percent similarity, and cost $0.014 per task with seven times fewer tokens than asking the model to back up itself. Link: https://arxiv.org/abs/2610.28934
The common thread fits on a sticky note. Do not trust a model to know its own limits, trust the structure around it: independent inputs, a rule layer that cannot be talked out of a no, a per-user threshold learned from the user, and an undo button the harness provides. Walmart's spending caps and Duet's hard no-go list from earlier this week are the product versions of exactly this.
Comments