METR and Redwood got inside OpenAI's agent monitor, and the holes were in the plumbing, not the model
METR and Redwood Research published the first external red-team of a frontier lab's internal agent-monitoring system on October 22. One person from each organization worked onsite at OpenAI from July 21 to August 11, on OpenAI devices, with access to the codebase, the monitor prompts and configs, and about a dozen staff interviews. The full output was a 61-page internal report. The public post is short and high-level by prior agreement, and it still has numbers. Link: https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/
Coverage first. Roughly 75 to 94 percent of agentic internal Responses API traffic was eligible for monitoring, and 99.2 percent of internal Codex traffic. RL training runs were not covered at all when the exercise started. Between May and July, unmonitored context compaction left on the order of 10 percent of eligible Codex traffic unseen. The monitor itself was accurate, with an error rate under 0.2 percent. The problem was what it got to look at.
The sharpest finding is about timing. The monitor fired when an agent handed control back to a human, so an agent could put off being monitored indefinitely by just continuing to act. The team also found a way for Codex agents to get around the sandbox and the Auto-Review step, and a way to hide hundreds of recent actions, with details redacted by OpenAI. OpenAI says the production-affecting issues are patched. CNBC coverage: https://www.cnbc.com/2026/10/22/metr-redwood-research-open-ai-test.html
This lands one day after OpenAI's own misalignment report described agents suppressing their alerts. The two documents tell the same story from opposite ends. The model being watched is not the weak point. Compaction, trigger timing and RL rollouts are, which means a monitor is only as good as the harness feeding it. And the evaluator here had badge access, escorts and an agreed redaction process. That is what independent oversight of a lab actually looks like in 2026: real, useful and still mostly behind an NDA.
← Back to all articles
Coverage first. Roughly 75 to 94 percent of agentic internal Responses API traffic was eligible for monitoring, and 99.2 percent of internal Codex traffic. RL training runs were not covered at all when the exercise started. Between May and July, unmonitored context compaction left on the order of 10 percent of eligible Codex traffic unseen. The monitor itself was accurate, with an error rate under 0.2 percent. The problem was what it got to look at.
The sharpest finding is about timing. The monitor fired when an agent handed control back to a human, so an agent could put off being monitored indefinitely by just continuing to act. The team also found a way for Codex agents to get around the sandbox and the Auto-Review step, and a way to hide hundreds of recent actions, with details redacted by OpenAI. OpenAI says the production-affecting issues are patched. CNBC coverage: https://www.cnbc.com/2026/10/22/metr-redwood-research-open-ai-test.html
This lands one day after OpenAI's own misalignment report described agents suppressing their alerts. The two documents tell the same story from opposite ends. The model being watched is not the weak point. Compaction, trigger timing and RL rollouts are, which means a monitor is only as good as the harness feeding it. And the evaluator here had badge access, escorts and an agreed redaction process. That is what independent oversight of a lab actually looks like in 2026: real, useful and still mostly behind an NDA.
Comments