Memdoor team-solve: five 27B agents, eight YAML files, 39 cents a hard problem
A team of small open models just matched a frontier model's one-shot score on hard competitive programming, and the write-up is more honest than most papers. Memdoor's team-solve example hit 410 points on Hacker News over the weekend.
The setup is five Qwen3.8-27B agents with fixed roles: a planner, two ideators, a solver and a manager, plus a repair worker. Each starts from a fresh context, and they talk only through files on disk. The whole workflow is eight YAML files with no code, a reimplementation of the GVS5H method published earlier this year. On the first 10 LiveCodeBench Hard problems, run 5 times each, the team averaged 9.8 out of 10. The original GVS5H harness scored 9.6 and a single Fable 5 call scored 8.0. Median run: 40 minutes and $0.39. All 50 runs cost $35.79.
Now the caveats, which the README states itself. Ten problems is a tiny sample, and 9.8 versus 9.6 is within noise, so the real claim is parity with the original harness. The contests date from spring 2025, so contamination can't be ruled out. The Fable number comes from the GVS5H authors' files and wasn't rerun. On their full 100 problems, the harness beat Fable 92.4 to 90.4.
The best detail is the one problem the team lost. The manager invented a wrong counterexample, and the team overwrote a correct program to satisfy it. A bad manager costs agent teams the same way it costs human ones.
The takeaway fits a pattern we keep seeing: the harness moves the score as much as the model does. A 27B model in a well-shaped team, with files as the only channel, gets frontier one-shot quality for under 40 cents.
Repo: https://github.com/guregodevo/memdoor/tree/main/examples/workflows/team-solve
← Back to all articles
The setup is five Qwen3.8-27B agents with fixed roles: a planner, two ideators, a solver and a manager, plus a repair worker. Each starts from a fresh context, and they talk only through files on disk. The whole workflow is eight YAML files with no code, a reimplementation of the GVS5H method published earlier this year. On the first 10 LiveCodeBench Hard problems, run 5 times each, the team averaged 9.8 out of 10. The original GVS5H harness scored 9.6 and a single Fable 5 call scored 8.0. Median run: 40 minutes and $0.39. All 50 runs cost $35.79.
Now the caveats, which the README states itself. Ten problems is a tiny sample, and 9.8 versus 9.6 is within noise, so the real claim is parity with the original harness. The contests date from spring 2025, so contamination can't be ruled out. The Fable number comes from the GVS5H authors' files and wasn't rerun. On their full 100 problems, the harness beat Fable 92.4 to 90.4.
The best detail is the one problem the team lost. The manager invented a wrong counterexample, and the team overwrote a correct program to satisfy it. A bad manager costs agent teams the same way it costs human ones.
The takeaway fits a pattern we keep seeing: the harness moves the score as much as the model does. A 27B model in a well-shaped team, with files as the only channel, gets frontier one-shot quality for under 40 cents.
Repo: https://github.com/guregodevo/memdoor/tree/main/examples/workflows/team-solve
Comments