Compile by Training: Turn the Prompt Into Local Weights, Then Stop Paying
The number one paper on HuggingFace's daily board this weekend, at 316 upvotes, is Compile by Training (https://arxiv.org/abs/2609.04199) from the University of Waterloo's Yuntian Deng and Pengyu Nie with Harvard's Stuart Shieber. The idea in one sentence: instead of calling a big remote model on every input forever, compile the natural-language spec once into a small local neural function, and call that instead.
The mechanics: at compile time, a teacher model generates task-specific training examples from your spec, which train a compact adapter on a small interpreter model. Compilation takes about a minute. What comes out is a standalone function — no remote calls, no per-token bill, no provider dependency, no rate limits. On FuzzyBench-Hard, the subset where a fast rule-based compiler baseline produced zero exact matches, the compiled functions hit 83.6% semantic accuracy. There's a public demo at https://programasweights.com, and the paper is headed to EMNLP 2026's demo track.
The target is every pipeline that calls an LLM per row for a fixed transformation — classify this, extract that, rewrite this — which is the dumbest and most common form of agent-loop cost. Same week Airtop's pitch was compiling browsing agents into deterministic scripts (https://clauday.com/article/71e9f22d-1d73-4773-b289-f68eb1bde143); Waterloo compiles specs into weights. The verb is spreading across layers, and "the prompt is the program" is turning literal: the program is weights now. For anything an agent does more than a few hundred times, compilation beats invocation — that's an old compiler-theory lesson arriving in the LLM bill.
← Back to all articles
The mechanics: at compile time, a teacher model generates task-specific training examples from your spec, which train a compact adapter on a small interpreter model. Compilation takes about a minute. What comes out is a standalone function — no remote calls, no per-token bill, no provider dependency, no rate limits. On FuzzyBench-Hard, the subset where a fast rule-based compiler baseline produced zero exact matches, the compiled functions hit 83.6% semantic accuracy. There's a public demo at https://programasweights.com, and the paper is headed to EMNLP 2026's demo track.
The target is every pipeline that calls an LLM per row for a fixed transformation — classify this, extract that, rewrite this — which is the dumbest and most common form of agent-loop cost. Same week Airtop's pitch was compiling browsing agents into deterministic scripts (https://clauday.com/article/71e9f22d-1d73-4773-b289-f68eb1bde143); Waterloo compiles specs into weights. The verb is spreading across layers, and "the prompt is the program" is turning literal: the program is weights now. For anything an agent does more than a few hundred times, compilation beats invocation — that's an old compiler-theory lesson arriving in the LLM bill.
Comments