Gemini 4 Carbon ships to cyber defenders first, and its safety section reads like an agent spec
Google launched Gemini 4 Carbon on October 28, and the first people who get to use it are not developers. "Starting this week, we will make Carbon available to a subset of cyber defenders," the announcement says, with wider access phased in through Google's Fairwind program and the model going through the US government's voluntary pre-release testing. Developers and consumers get it "as soon as possible." That makes two frontier launches in a row that went to defenders before the public.
The headline number is 59 on the Artificial Analysis Intelligence Index v5, which puts Carbon at the top of an index that was rebuilt a day earlier so that 60% of it runs on private or held-out test sets. On security work it posts 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench, and the Chrome team says Carbon did "more than 10X better than another lab's large frontier model when generating high quality patches." Google also says internal teams have seen Carbon "run for days to persist through dead ends." The launch price matches Argon at $2 per million input tokens and $10 per million output, with cached input 95% off, but that is introductory: it goes to $4 and $20 afterward. Artificial Analysis ran its own numbers the same afternoon and confirmed the 59, measured $7.76 per index task against $10.04 for Opus 5.5 at max effort, and put Carbon three points behind Claude Code with Opus 5.5 on its coding-agent index.
The section worth reading slowly is the safety one, because every line is about how an agent behaves on a long task. Successful cheating on Terminal-Bench 4.0 is down to 0.0%, and attempts to cheat to 0.4%. High-severity misalignment alerts in live coding with internal teams dropped by almost 80%. Unsupported claims are cut roughly in half versus Argon, though Artificial Analysis measured its hallucination rate rising from Argon's 4% to 11%, still second-lowest of any model it tracks. Overreach, doing more than you were asked, is down 26%, and under-service, quietly doing less, is down 29%. Gray Swan's indirect prompt injection success went from 0.7% to 0.0%.
A year ago a model card listed refusal rates and toxicity scores. Carbon's lists cheating, overreach, under-service and fabrication, each with a number. That is the checklist anyone running agents in production already keeps by hand. When a lab starts competing on those four numbers, the agent-reliability conversation has moved from blog posts into the launch announcement. The open question is whether the numbers hold up once people outside Google run Carbon on their own tasks.
Announcement: https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/
← Back to all articles
The headline number is 59 on the Artificial Analysis Intelligence Index v5, which puts Carbon at the top of an index that was rebuilt a day earlier so that 60% of it runs on private or held-out test sets. On security work it posts 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench, and the Chrome team says Carbon did "more than 10X better than another lab's large frontier model when generating high quality patches." Google also says internal teams have seen Carbon "run for days to persist through dead ends." The launch price matches Argon at $2 per million input tokens and $10 per million output, with cached input 95% off, but that is introductory: it goes to $4 and $20 afterward. Artificial Analysis ran its own numbers the same afternoon and confirmed the 59, measured $7.76 per index task against $10.04 for Opus 5.5 at max effort, and put Carbon three points behind Claude Code with Opus 5.5 on its coding-agent index.
The section worth reading slowly is the safety one, because every line is about how an agent behaves on a long task. Successful cheating on Terminal-Bench 4.0 is down to 0.0%, and attempts to cheat to 0.4%. High-severity misalignment alerts in live coding with internal teams dropped by almost 80%. Unsupported claims are cut roughly in half versus Argon, though Artificial Analysis measured its hallucination rate rising from Argon's 4% to 11%, still second-lowest of any model it tracks. Overreach, doing more than you were asked, is down 26%, and under-service, quietly doing less, is down 29%. Gray Swan's indirect prompt injection success went from 0.7% to 0.0%.
A year ago a model card listed refusal rates and toxicity scores. Carbon's lists cheating, overreach, under-service and fabrication, each with a number. That is the checklist anyone running agents in production already keeps by hand. When a lab starts competing on those four numbers, the agent-reliability conversation has moved from blog posts into the launch announcement. The open question is whether the numbers hold up once people outside Google run Carbon on their own tasks.
Announcement: https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/
Comments