3,000 merged changes in two weeks, and the lesson is about measurement
Anthropic made claude.ai and the desktop app about 3x faster in a two-week sprint in August. The interesting part isn't the speedup, it's that a model merged over 3,000 changes across roughly 150 concurrent threads with zero customer-facing incidents and zero rollbacks. Peak days landed 200+ changes.
The numbers, 75th percentile, August 13 versus August 27. Fresh claude.ai load 3,085ms to 550ms, a 5.6x cut. Desktop cold start 6,310ms to 3,328ms. Claude Code desktop start 837ms to 347ms. Loading a conversation 1,557ms to 646ms. Cowork cloud session 2,566ms to 728ms. Sending a message in Cowork 928ms to 48ms, which is 19x. Twelve of thirteen initial targets were hit by day three.
The loop is the thing to copy. An engineer or the model spots a slow journey from a screenshot or recording. The model then builds a reproducible measurement first — Valgrind instruction counts, V8 call counts, React commits, style recalculations, DOM mutations. Only then does it prototype, submitting multiple PRs sized for risk, user-visible changes behind flags. Ship to employees, then 1%, then everyone. Read field data from real users organized by build and platform. On success, ratchet the benchmark down as a permanent guardrail. On failure, kill the flag and keep digging. Nearly 200 flags created, over half already retired by the end.
The sentence they built the post around: once Claude can measure something, it can make it faster. Which means the highest-leverage human work was finding more things to measure. That inverts how most people use coding agents, where the human specifies the change and the agent writes it. Here the human specifies the metric and the agent finds the change.
The model was Claude Tag, an internal research model roughly at Opus 5.5 level, operating out of a single Slack channel with standing instructions. Three human steering principles: ambition (push past the model's conservative scope), taste (humans decide user-perceptible tradeoffs), direction (keep threads narrow and sequenced). An engineer's reaction — you could not have convinced me this was possible even six months ago — is worth more than the benchmark table.
The caveat: performance work is the friendliest possible domain for this, because the success criterion is a number that can't be argued with. That is exactly why it worked, and exactly why it doesn't generalize to everything.
https://claude.dev/blog/how-we-made-claude-ai-faster/
← Back to all articles
The numbers, 75th percentile, August 13 versus August 27. Fresh claude.ai load 3,085ms to 550ms, a 5.6x cut. Desktop cold start 6,310ms to 3,328ms. Claude Code desktop start 837ms to 347ms. Loading a conversation 1,557ms to 646ms. Cowork cloud session 2,566ms to 728ms. Sending a message in Cowork 928ms to 48ms, which is 19x. Twelve of thirteen initial targets were hit by day three.
The loop is the thing to copy. An engineer or the model spots a slow journey from a screenshot or recording. The model then builds a reproducible measurement first — Valgrind instruction counts, V8 call counts, React commits, style recalculations, DOM mutations. Only then does it prototype, submitting multiple PRs sized for risk, user-visible changes behind flags. Ship to employees, then 1%, then everyone. Read field data from real users organized by build and platform. On success, ratchet the benchmark down as a permanent guardrail. On failure, kill the flag and keep digging. Nearly 200 flags created, over half already retired by the end.
The sentence they built the post around: once Claude can measure something, it can make it faster. Which means the highest-leverage human work was finding more things to measure. That inverts how most people use coding agents, where the human specifies the change and the agent writes it. Here the human specifies the metric and the agent finds the change.
The model was Claude Tag, an internal research model roughly at Opus 5.5 level, operating out of a single Slack channel with standing instructions. Three human steering principles: ambition (push past the model's conservative scope), taste (humans decide user-perceptible tradeoffs), direction (keep threads narrow and sequenced). An engineer's reaction — you could not have convinced me this was possible even six months ago — is worth more than the benchmark table.
The caveat: performance work is the friendliest possible domain for this, because the success criterion is a number that can't be argued with. That is exactly why it worked, and exactly why it doesn't generalize to everything.
https://claude.dev/blog/how-we-made-claude-ai-faster/
Comments