Agents Double-Charge 74% of the Time When the Network Lies
A payment call times out. Did the charge go through? Retry and you might bill the customer twice. Give up and you might not bill them at all. Every backend engineer knows this problem. A paper posted September 24 by Jiapeng Li asks where agents should solve it, in the model, the harness, or the tool contract, and the answer is that it depends entirely on what the agent can see.
The testbed, LIMBO, is a deterministic sandbox of six services with realistic contracts and twelve injected fault modes, including late commits, redelivery and partial batches. Every episode is graded against a ledger of what actually got committed. 25,930 episodes, nine recent models, three production agent harnesses. When an immediate read-back can reveal whether the write landed, the model is what matters: frontier models told to act exactly once almost never duplicate, 0.5 percent, while weaker models often do. When the read-back cannot tell, because the request is still in flight or the transport delivered it twice, the same frontier models duplicate in 56 and 74 percent of episodes. There, the tool contract explains 81 percent of the variance and the model barely matters.
The author also proves no verification-only policy can be exactly-once under late commits without a bound on in-flight time. Waiting helps when that bound is short and known, but with heavy-tailed delays even an hour of waiting per episode is not enough.
The takeaway for anyone wiring agents to real systems: stop trying to prompt your way to idempotency. If the API does not take an idempotency key, no amount of model intelligence saves you when the network misbehaves. Smarter models fix the easy half. Contracts fix the hard half. Paper: arxiv.org/abs/2609.29095.
← Back to all articles
The testbed, LIMBO, is a deterministic sandbox of six services with realistic contracts and twelve injected fault modes, including late commits, redelivery and partial batches. Every episode is graded against a ledger of what actually got committed. 25,930 episodes, nine recent models, three production agent harnesses. When an immediate read-back can reveal whether the write landed, the model is what matters: frontier models told to act exactly once almost never duplicate, 0.5 percent, while weaker models often do. When the read-back cannot tell, because the request is still in flight or the transport delivered it twice, the same frontier models duplicate in 56 and 74 percent of episodes. There, the tool contract explains 81 percent of the variance and the model barely matters.
The author also proves no verification-only policy can be exactly-once under late commits without a bound on in-flight time. Waiting helps when that bound is short and known, but with heavy-tailed delays even an hour of waiting per episode is not enough.
The takeaway for anyone wiring agents to real systems: stop trying to prompt your way to idempotency. If the API does not take an idempotency key, no amount of model intelligence saves you when the network misbehaves. Smarter models fix the easy half. Contracts fix the hard half. Paper: arxiv.org/abs/2609.29095.
Comments