Your LLM judge flips one verdict in five, even at temperature zero
Four papers from last Friday's arXiv batch point at the same weak spot: the model doing the grading. If you use an LLM judge as a reward signal or as a pass/fail gate, all four are worth reading.
The cleanest one reran Arena-Hard v2.0. The authors took 500 prompts, used GPT-4.1 as the judge on a pinned model version at temperature zero, and simply ran the default setup again. The aggregate score moved about one point, which looks stable. But the win/tie/loss outcome changed on 21.2% of prompts. Turning off the judge's reasoning changed outcomes on another 28.2 points' worth of prompts and dropped the aggregate by 4.09. Their summary: "A stable aggregate score therefore does not establish that individual verdicts are repeatable."
The second asks whether a judge that grades more accurately also trains better. Not necessarily. Across Qwen3 reward models from 0.6B to 14B, "Thinking" judges trained with GRPO beat answer-only judges on static accuracy, but when each was used to train policies with online DPO, the answer-only judges produced better policies in seven of eight matched settings. Intermediate checkpoints sometimes made better rewarders than later, more accurate ones. The authors call it the gap between evaluator and rewarder, and caveat that the setting is creative writing only.
The other two make the same point from the product side. TauIndianBankBench, from India's payments operator NPCI, grades 1,000 retail banking tasks on whether every required step was taken and nothing unrequested was changed. For every model, 6.5% to 12.0% of sessions look successful if you check only the final records. And a 27-language study with over a million scores found that whether a prompting strategy helps can flip direction depending on which judge you ask.
Taken together, the risk is clear. Per-item LLM verdicts are noisy labels, leaderboard accuracy doesn't certify a reward model, and checking only the end state hides process failures. If an agent's RL loop or CI gate leans on a single judge call, it is training on a coin that lands differently about one time in five.
Arena-Hard study: https://arxiv.org/abs/2610.30512 (code https://github.com/Yorkel/llm-judge-release) GenRM scaling: https://arxiv.org/abs/2610.30765 TauIndianBankBench: https://arxiv.org/abs/2610.30161 (code https://github.com/npci/TauIndianBankBench) Judge-dependent prompting: https://arxiv.org/abs/2610.31172
← Back to all articles
The cleanest one reran Arena-Hard v2.0. The authors took 500 prompts, used GPT-4.1 as the judge on a pinned model version at temperature zero, and simply ran the default setup again. The aggregate score moved about one point, which looks stable. But the win/tie/loss outcome changed on 21.2% of prompts. Turning off the judge's reasoning changed outcomes on another 28.2 points' worth of prompts and dropped the aggregate by 4.09. Their summary: "A stable aggregate score therefore does not establish that individual verdicts are repeatable."
The second asks whether a judge that grades more accurately also trains better. Not necessarily. Across Qwen3 reward models from 0.6B to 14B, "Thinking" judges trained with GRPO beat answer-only judges on static accuracy, but when each was used to train policies with online DPO, the answer-only judges produced better policies in seven of eight matched settings. Intermediate checkpoints sometimes made better rewarders than later, more accurate ones. The authors call it the gap between evaluator and rewarder, and caveat that the setting is creative writing only.
The other two make the same point from the product side. TauIndianBankBench, from India's payments operator NPCI, grades 1,000 retail banking tasks on whether every required step was taken and nothing unrequested was changed. For every model, 6.5% to 12.0% of sessions look successful if you check only the final records. And a 27-language study with over a million scores found that whether a prompting strategy helps can flip direction depending on which judge you ask.
Taken together, the risk is clear. Per-item LLM verdicts are noisy labels, leaderboard accuracy doesn't certify a reward model, and checking only the end state hides process failures. If an agent's RL loop or CI gate leans on a single judge call, it is training on a coin that lands differently about one time in five.
Arena-Hard study: https://arxiv.org/abs/2610.30512 (code https://github.com/Yorkel/llm-judge-release) GenRM scaling: https://arxiv.org/abs/2610.30765 TauIndianBankBench: https://arxiv.org/abs/2610.30161 (code https://github.com/npci/TauIndianBankBench) Judge-dependent prompting: https://arxiv.org/abs/2610.31172
Comments