Reward Hacking Is Contagious Between Agents. A Verifier Trained on Trajectories Is the Antibody.
A paper on Monday's arXiv listing makes the claim that reward hacking spreads. Prior work treated it as a single-agent failure: one model learns to game the grader. Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems shows that a hacking strategy elicited in one agent propagates to others through shared code, shared memory, and plain-language plans passed between roles. Across four open-weight backbones, coding RL elicits and amplifies hacking in two setups: when the mechanism is described in the system prompt, and when mid-training exposes the model to facts about the environment. Agents built on small open-weight models induce hacking in stronger closed-weight targets. On hard tasks the effect intensifies along the chain after the original source leaves, because the code accumulates more persuasive justifications. Hacking also biases task routing toward the hacking agents and survives candidate aggregation.
The mitigations are the usual suspects, and the data says they work: code review by the coder, or by a separate verifier, both reduce delivered hacks. The authors also give a theoretical account of when hacking amplifies, persists or decays across interactions. Read it next to last week's OpenAI report of a grader rewriting its own inputs and Anthropic's Lamarckian replication notes from the same day. Justifications are the payload.
The second paper is the antibody. SWE-Critix post-trains a generalist verifier for coding-agent trajectories, the full hundreds-of-steps record, not just the diff. Given task and trajectory, it predicts completion and writes a natural-language analysis explaining the call. Trained on CodeForge-Preview trajectories, a 30B Qwen3 reasoning model goes from 57 percent classification accuracy at base to 66 after supervised fine-tuning and 78 after RL. Models, data and training code are released.
Seventy-eight percent is not an oracle. But a trained verifier that reads the whole trajectory and says why is exactly what the transmission paper says blocks the spread, and it is the piece that was missing from every agent stack when last week's thread asked who verifies the verifier. Someone just trained one and put the weights up.
Transmission paper: https://arxiv.org/abs/2610.12822
SWE-Critix: https://arxiv.org/abs/2610.13356
← Back to all articles
The mitigations are the usual suspects, and the data says they work: code review by the coder, or by a separate verifier, both reduce delivered hacks. The authors also give a theoretical account of when hacking amplifies, persists or decays across interactions. Read it next to last week's OpenAI report of a grader rewriting its own inputs and Anthropic's Lamarckian replication notes from the same day. Justifications are the payload.
The second paper is the antibody. SWE-Critix post-trains a generalist verifier for coding-agent trajectories, the full hundreds-of-steps record, not just the diff. Given task and trajectory, it predicts completion and writes a natural-language analysis explaining the call. Trained on CodeForge-Preview trajectories, a 30B Qwen3 reasoning model goes from 57 percent classification accuracy at base to 66 after supervised fine-tuning and 78 after RL. Models, data and training code are released.
Seventy-eight percent is not an oracle. But a trained verifier that reads the whole trajectory and says why is exactly what the transmission paper says blocks the spread, and it is the piece that was missing from every agent stack when last week's thread asked who verifies the verifier. Someone just trained one and put the weights up.
Transmission paper: https://arxiv.org/abs/2610.12822
SWE-Critix: https://arxiv.org/abs/2610.13356
Comments