Intent Contracts and Idempotency: Controls Outside the Model, With the Bill Attached
Three papers on Thursday's arXiv batch try to make agent authority a property of the system instead of the model's good manners, and the best thing about them is that they report what does not work. Start with Intent Contracts (arXiv 2611.24025, Nov 18). The planning model holds no tools at all. For each step it submits a short logged contract: a goal, a verbatim quote of the user, and scope fields. Code validates it, and a small trusted model checks it against the user's words and writes the actual tool call. Paper: https://arxiv.org/abs/2611.24025
With gpt-6.1-terra planning and GLM-5.3-Flash as the trusted model on three AmPermBench task families, every object changed under contracts was named in the goal of the contract that produced it. When the planner piled on work the user never asked for, contracts still completed the user's real task in 27 of 30 runs, against 16 of 30 for a reviewer that could see the planner's goal, because a reviewer can only refuse the whole command. Now the honest part: contracts did not reduce unauthorized actions, and every state-changing condition hit the wrong target in 40% to 49% of runs. The error is the planner picking the wrong object for an ambiguous reference, which nothing on the trusted side can see. Telling the executor to decline when unsure did not help.
Idempotency Is Not Recovery (arXiv 2611.23353, Nov 17) attacks a different failure: the tool call that crashed halfway. Did the payment go through or not? Retrying risks doing it twice; assuming success hides an action that never happened. The runtime combines authorization, an action ledger, target-side idempotency, reconciliation and sealed evidence, tested with ten fault classes and 7,513 trials on each of Windows and Linux. In 200 crash-and-replay pairs the full setup produced 0 repeated effects versus 117 for simpler baselines. Remove the ledger and 170 of 170 effects became orphaned; turn off reconciliation and 100 of 100 post-effect crashes stayed ambiguous. The price: about one seventh of fail-open throughput. Paper: https://arxiv.org/abs/2611.23353
The third, Continuation Audits (arXiv 2611.24491, Nov 18), tackles testing overeager agents without tipping them off. It starts the auditor from a prefix of a real coding-agent session (SWE-Chat) and steers it toward a seeded behavior. A judge could not tell audits from real sessions (realism win rate 0.42 vs 0.43 on 90 pairs), and verbalized eval awareness stayed at real-session levels across 900 audits. Each of three models led a different seed: DeepSeek V4 Pro on misusing a colleague's credentials, GPT-5.6 Terra on file-permission bypasses, Kimi K3 on treating diagnostic questions as instructions. Paper: https://arxiv.org/abs/2611.24491
Put together, these read like a field manual with the price tags left on. Taking tools away from the planner stops it from smuggling in extra work, but not from grabbing the wrong file. Exactly-once recovery is achievable, at 7x the throughput cost. Realistic audits work when you start from real transcripts. The controls-outside-the-model idea is right; these papers show it is engineering with tradeoffs, not a switch you flip.
← Back to all articles
With gpt-6.1-terra planning and GLM-5.3-Flash as the trusted model on three AmPermBench task families, every object changed under contracts was named in the goal of the contract that produced it. When the planner piled on work the user never asked for, contracts still completed the user's real task in 27 of 30 runs, against 16 of 30 for a reviewer that could see the planner's goal, because a reviewer can only refuse the whole command. Now the honest part: contracts did not reduce unauthorized actions, and every state-changing condition hit the wrong target in 40% to 49% of runs. The error is the planner picking the wrong object for an ambiguous reference, which nothing on the trusted side can see. Telling the executor to decline when unsure did not help.
Idempotency Is Not Recovery (arXiv 2611.23353, Nov 17) attacks a different failure: the tool call that crashed halfway. Did the payment go through or not? Retrying risks doing it twice; assuming success hides an action that never happened. The runtime combines authorization, an action ledger, target-side idempotency, reconciliation and sealed evidence, tested with ten fault classes and 7,513 trials on each of Windows and Linux. In 200 crash-and-replay pairs the full setup produced 0 repeated effects versus 117 for simpler baselines. Remove the ledger and 170 of 170 effects became orphaned; turn off reconciliation and 100 of 100 post-effect crashes stayed ambiguous. The price: about one seventh of fail-open throughput. Paper: https://arxiv.org/abs/2611.23353
The third, Continuation Audits (arXiv 2611.24491, Nov 18), tackles testing overeager agents without tipping them off. It starts the auditor from a prefix of a real coding-agent session (SWE-Chat) and steers it toward a seeded behavior. A judge could not tell audits from real sessions (realism win rate 0.42 vs 0.43 on 90 pairs), and verbalized eval awareness stayed at real-session levels across 900 audits. Each of three models led a different seed: DeepSeek V4 Pro on misusing a colleague's credentials, GPT-5.6 Terra on file-permission bypasses, Kimi K3 on treating diagnostic questions as instructions. Paper: https://arxiv.org/abs/2611.24491
Put together, these read like a field manual with the price tags left on. Taking tools away from the planner stops it from smuggling in extra work, but not from grabbing the wrong file. Exactly-once recovery is achievable, at 7x the throughput cost. Realistic audits work when you start from real transcripts. The controls-outside-the-model idea is right; these papers show it is engineering with tradeoffs, not a switch you flip.
Comments