CodeMidas turns 3,185 random repos into an RL gym for coding agents
Training a coding agent with reinforcement learning needs two things that are hard to get together: diverse tasks, and verifiers you can trust. CodeMidas, arXiv 2609.22068, submitted September 18 and sitting at 86 upvotes on HuggingFace's board today, solves it by mining open-source code and turning it into executable RL environments automatically.
The pipeline uses agents to do the boring part. One formulates a behavioral specification from a chunk of code, another constructs tests against that spec, another validates that the resulting task is actually solvable and actually checks something. Output: 5,545 training tasks drawn from 3,185 codebases across 23 languages and 15 domains. Every task ships with a verifier, because the verifier is generated from the same source as the task.
The gains transfer and that is the part worth noting. DeepSWE on issue repair improves 11.7 percent. ProgramBench, whole-program construction, improves 17 percent. Terminal-Bench v2.1, terminal work, improves 8.5 percent. Three different task shapes, three different benchmarks, none of them the training distribution. Whole-program construction gaining the most makes sense, since building something from a spec is closest to what the environments actually teach.
The framing that matters here is about supply. Everyone agrees agent RL works. The bottleneck has always been environments, which are expensive because someone has to write the task and then write the thing that decides whether you passed. Companies build these by hand and treat them as moats. CodeMidas's claim is that GitHub already contains the environments, and all you need is a pipeline that reads code and emits a spec plus tests.
Put it next to the paper trend of the last month and the shape gets clearer: build the environment and a smaller model becomes useful, rather than build a bigger model and hope. What I want from this one and did not find on the abstract page is whether the environments themselves are released. The 5,545 tasks are the asset. The method is a paper; the gym is a public good, and those are very different contributions. https://arxiv.org/abs/2609.22068
← Back to all articles
The pipeline uses agents to do the boring part. One formulates a behavioral specification from a chunk of code, another constructs tests against that spec, another validates that the resulting task is actually solvable and actually checks something. Output: 5,545 training tasks drawn from 3,185 codebases across 23 languages and 15 domains. Every task ships with a verifier, because the verifier is generated from the same source as the task.
The gains transfer and that is the part worth noting. DeepSWE on issue repair improves 11.7 percent. ProgramBench, whole-program construction, improves 17 percent. Terminal-Bench v2.1, terminal work, improves 8.5 percent. Three different task shapes, three different benchmarks, none of them the training distribution. Whole-program construction gaining the most makes sense, since building something from a spec is closest to what the environments actually teach.
The framing that matters here is about supply. Everyone agrees agent RL works. The bottleneck has always been environments, which are expensive because someone has to write the task and then write the thing that decides whether you passed. Companies build these by hand and treat them as moats. CodeMidas's claim is that GitHub already contains the environments, and all you need is a pipeline that reads code and emits a spec plus tests.
Put it next to the paper trend of the last month and the shape gets clearer: build the environment and a smaller model becomes useful, rather than build a bigger model and hope. What I want from this one and did not find on the abstract page is whether the environments themselves are released. The 5,545 tasks are the asset. The method is a paper; the gym is a public good, and those are very different contributions. https://arxiv.org/abs/2609.22068
Comments