November 21, 2026InfrastructureMonitoringResearch

Two Million Canaries: Your Cheap Inference Route Is Reading Your Prompts

Every hop between your app and the model can read the whole prompt. Dragos just measured what that means in practice. Over five months, from July through November, a team led by Dragos chief research officer HD Moore sent nearly two million prompts to AI services, each carrying unique secrets that would never be used legitimately: real AWS keys tied to empty, fully logged accounts, SSH passwords, one-off URLs, DNS names, private git references. Then they waited to see which ones came back to life. The post went up November 19 at 13:00 UTC; Ars Technica's write-up landed November 20. Post: https://www.dragos.com/blog/ai-supply-chain-security-canary-tokens

Three findings. First, at least three hosting providers were breached and did not know until Dragos told them; the earliest incident dates to mid-July. In the largest, a couple hundred bait AWS keys were stolen, and every request sent during a few late-October windows was taken, including requests the provider rejected before any model ran, which means the copying happened at the front door. In the other two, some stolen requests carried the Zero Data Retention flag. The bait keys were then used for tens of thousands of AWS API calls, including attempts to switch on hosted models in Bedrock.

Second, and more depressing: no exploit was needed. Some sellers on an open inference marketplace simply listed a popular model at the lowest price, turned on request logging and waited. Individuals on home connections then tried the leaked URLs and passwords by hand. The same model through a different seller stayed clean, so the leak followed the seller, not the marketplace. Third, privacy settings were ignored outright. A Chinese AI lab's systems fetched secret URLs from more than half the prompts sent to its models, including about two thirds of those marked no data collection or ZDR, and prompts sent to US hosts reselling that lab's models triggered the same scraper. One secret link surfaced in a company chat 24 days after a ZDR request. Dragos does not name the breached providers.

This is the bill for the cost-per-task era. Routers and resellers are how teams cut inference spend, and the cheapest seller in a pool wins the request. Dragos also built a sort of traceroute for inference and found several hundred of about two thousand traced routes included a system it could not identify. Moore's advice, per Ars: self-host if you can afford it, otherwise buy directly from a first-tier provider with one contract and one privacy policy, and pick your upstream yourself instead of accepting a default pool. Ars write-up: https://arstechnica.com/security/2026/11/a-leaky-ai-supply-chain-is-exposing-secret-user-data-at-an-alarming-rate/

My read: a ZDR flag is a request, not a control. If an agent pipeline puts secrets in prompts, assume every hop keeps a copy, and seed your own canaries to find out which one talks.
← Previous
Signal Treats the OS Agent as the Adversary
Next β†’
Models Engineering Index: Neon Wins, the Cost Column Decides
← Back to all articles

Comments

Loading...
>_