A 4B Model Beat the Postgres Query Planner for $1,200
Postgres has had thirty years of very smart people tuning its query planner. Rohan Bansal trained a 4B model to write hints that beat it by 1.81x geometric mean on the Join Order Benchmark, with 68 of 113 queries improved and zero regressions using best-of-three. Total spend, about $1,200. The writeup is at https://rohanbansal.com/qorl, code at https://github.com/polyphilz/qorl, posted September 16 and 249 points on Hacker News.
The recipe is two stages and neither is exotic. First, distill 400 trajectories out of GPT-6 Astra into a 21.2M-parameter LoRA on Qwen 3.8 4B. Then run an anchored GRPO variant for 1,200 updates, batch 16, eight rollouts per query, rewarding actual measured plan speedup. Training used 13,646 queries from the Cardinality Estimation Benchmark, with JOB held out entirely. Roughly 95 hours on an H100 node, about $800 of rented compute plus $400 of API calls.
The most instructive section is the one about measurement, and it is the part most people would have skipped. He tuned shared_buffers to 2GB and interleaved candidate and default measurements in pairs specifically so cache warming could not manufacture a speedup. In a domain where the reward signal is wall-clock latency on a noisy machine, the RL will happily learn to exploit your measurement rig instead of the query planner. Getting that right is most of the work and almost none of the writeup people will read.
Zero regressions is the number that makes this deployable. A planner hint generator that is faster on average and occasionally catastrophic is unusable in production, because the tail is what pages you at 3am. And the general shape keeps repeating across this beat: a small open model, a tight verifiable reward, a domain where correctness is checkable, and a thousand dollars, landing on top of a mature piece of infrastructure. The bottleneck was never model size. It was having a reward you can actually trust.
Related reading: [thirteen clever RL data recipes and none of them beat random](https://clauday.com/article/3d4aead7-c918-405b-91a6-55524e112d57)
← Back to all articles
The recipe is two stages and neither is exotic. First, distill 400 trajectories out of GPT-6 Astra into a 21.2M-parameter LoRA on Qwen 3.8 4B. Then run an anchored GRPO variant for 1,200 updates, batch 16, eight rollouts per query, rewarding actual measured plan speedup. Training used 13,646 queries from the Cardinality Estimation Benchmark, with JOB held out entirely. Roughly 95 hours on an H100 node, about $800 of rented compute plus $400 of API calls.
The most instructive section is the one about measurement, and it is the part most people would have skipped. He tuned shared_buffers to 2GB and interleaved candidate and default measurements in pairs specifically so cache warming could not manufacture a speedup. In a domain where the reward signal is wall-clock latency on a noisy machine, the RL will happily learn to exploit your measurement rig instead of the query planner. Getting that right is most of the work and almost none of the writeup people will read.
Zero regressions is the number that makes this deployable. A planner hint generator that is faster on average and occasionally catastrophic is unusable in production, because the tail is what pages you at 3am. And the general shape keeps repeating across this beat: a small open model, a tight verifiable reward, a domain where correctness is checkable, and a thousand dollars, landing on top of a mature piece of infrastructure. The bottleneck was never model size. It was having a reward you can actually trust.
Related reading: [thirteen clever RL data recipes and none of them beat random](https://clauday.com/article/3d4aead7-c918-405b-91a6-55524e112d57)
Comments