Recursive Self-Improvement: The Complete Mid-2026 Map — What RSI Actually Is, Who Builds It, How It's Measured, Where the Red Lines Sit
Recursive Self-Improvement (RSI) is one of the most-cited, least-defined terms in AI. The literal meaning is simple: AI improves AI, the improved AI improves the next version, and the loop compounds. I.J. Good finished the thought experiment in 1965 — "the first ultraintelligent machine is the last invention that man need ever make." For sixty years it stayed a philosophy problem. In the past two years it abruptly became an engineering one: it now has a formal definition, working systems, dedicated benchmarks, red lines written into lab safety policies, and startups valued on it. This piece covers what the field actually is, who is in it, and how far it has gotten. (This is the sister piece to "AI Building AI": that one covered how AI currently participates in AI production; this one covers the self-improvement loop itself — concept, routes, companies, measurement, red lines, debates.)
1. First, the definition: the field only got formal language in 2026
For years the definition of RSI belonged to LessWrong and science fiction. In July 2026 that changed: a survey from Schmidhuber's group (arXiv 2607.13104, posted July 14) formalized self-improvement for the first time as a clean concept — the self-induced update operator: a system that obtains and commits updates to itself. There are only two things to update, and the entire technical map of the field hides in this split: model parameters (θ→θ+1 — self-generated training data, self-evaluative feedback, exploratory experience) or the scaffold (Σ→Σ+1 — prompt optimization, memory, tool creation, full rewrite of the agent's own code). The flagship of the former is MIT's SEAL; of the latter, Sakana's Darwin Gödel Machine — both below.
The same month, another survey (arXiv 2607.07663, July 8) did something more concrete: it read and classified roughly 1,250 relevant papers from 2024–2026. Its conclusion is the field's true status: "the literature's mass sits where a human still audits the loop." The distribution: 393 papers on deployment-time self-refinement, 340 on training bootstraps, 318 on self-evaluation — versus only 139 touching automated research and 60 touching safety. About 74% of the papers are from 2026 — the field is exploding, but what's exploding is "weak RSI" (human-in-the-loop, bounded, evaluable self-improvement). Genuinely closed-loop self-modification remains rare. That is the baseline for reading through all the hype: today's RSI is a field full of human guardrails, and "strong RSI" (unbounded, closed-loop, self-directed) still exists only as the hypothetical adversary in safety policies.
2. Two technical pathways, each with one working flagship
The scaffold pathway's flagship is the Darwin Gödel Machine (DGM, Sakana AI with Jeff Clune's UBC group, ICLR 2026 poster). The name comes from Schmidhuber's 2003 Gödel Machine — the theoretical construct requiring a formal proof of benefit before any self-modification, which nobody built in sixty years because such proofs are impossible in practice. DGM's core move is swapping proof for evidence: the agent iteratively modifies its own code, validates each change empirically on coding benchmarks, and maintains an archive of agents for Darwinian open-ended exploration — keeping the whole branching tree rather than just the best variant, mutating from any node. Results: self-modified code took SWE-bench from 20.0% to 50.0% and Polyglot from 14.2% to 30.7% — more than doubling — and the improvements it discovered on its own (better code-editing tools, long-context management, a peer-review mechanism) are exactly what human engineers would build, except nobody told it to. Two caveats: it modifies the agent scaffold while the underlying model weights stay frozen, and it used a SWE-bench subset. On the same pathway, the earlier STOP (recursive scaffold self-improvement with a frozen LLM) and ADAS (automated design of agentic systems) were the academic prelude; DGM is the current synthesis.
The parameter pathway's flagship is MIT's SEAL (NeurIPS 2025). The model generates its own finetuning data and update directives ("self-edits"), which become persistent weight updates via SFT — and crucially, the generation policy itself is trained with RL, where the reward is the downstream performance of the updated model. The model is learning how to teach itself. The signature number: on knowledge incorporation, Qwen2.5-7B using self-generated data lifted no-context SQuAD from 33.5% to 47.0%, beating synthetic data generated by GPT-4.1 (46.3%) — a 7B model's RL-optimized self-curriculum outperforming a frontier model's curriculum written for it. The 0.7-point margin is small; the directional meaning is not: the optimal supplier for the data flywheel may be the model itself.
The third, evolutionary route (FunSearch, AlphaEvolve: LLM as mutation operator, evolving codebases) was covered in detail in the sister piece; one positioning note here: it is the only route in production — AlphaEvolve recovers 0.7% of Google's global compute and sped up a key Gemini training kernel by 23%, running in production for over a year. Assemble the three routes and you get the state of play: the evolutionary route ships, the scaffold route self-bootstraps best, the parameter route comes closest to the literal meaning of "changing oneself" — and all three stay inside the safety zone where the evaluator has the final word.
3. The evaluator is the ceiling of the whole game
The sharpest insight in that 1,250-paper survey deserves its own section: demonstrated self-improvement strength decays strictly along a verification hierarchy. Where there's a formal verifier (mathematical proof, a compiler), iteration can run indefinitely. Where there's only execution feedback (does the code run), improvement is bounded. Where the evaluator is a learned judge, the judge degrades across iterations. Where the signal is the model's own self-assessment, it is most easily gamed. In one sentence: an RSI loop's ceiling is exactly the quality of its evaluator.
This is not an abstract worry. The self-rewarding route (generator and evaluator sharing weights) has a structural failure mode confirmed independently by multiple papers: the loop systematically over-rewards high-confidence mistakes — the errors the model is most sure about get reinforced first, and this requires no adversary; it is the default. Work like CREAM exists specifically to patch this (consistency regularization), and the existence of the patch literature is itself proof of the disease. Put this next to the verification hierarchy and you understand why every working RSI system today lives in code and math: those are the only two domains with cheap, reliable, unbribable evaluators.
4. The AI Scientist reaches Nature: a milestone, and its asterisks
The automated-research-agent line got its institutional certification on March 26, 2026: Sakana AI's The AI Scientist (with UBC, Vector Institute, and Oxford) was published as a peer-reviewed paper in Nature (Lu et al., Nature 651:914–919). Read it precisely: what reached Nature is the methods paper describing the system — human authors, human reviewers — not an AI-written paper passing Nature review. The AI-written papers have a separate scoreboard: AI Scientist-v2 produced the first fully AI-generated paper to pass human peer review, scoring an average 6.33 (6/7/6) in double-blind review at the ICLR 2025 ICBINB workshop — above the human acceptance threshold, better than 55% of human submissions. The asterisks, reported honestly: workshop bars are far lower than main conferences; the paper was withdrawn as pre-planned; it contained citation errors; Sakana itself concedes none of its three submissions met their internal main-conference standard. External critics go further — one analysis claims roughly 57% of v2's generated papers contain fabricated data. Sakana also offered a self-serving observation: as the underlying foundation models improve, generated-paper quality rises correspondingly — what they call "a scaling law for AI science." If true, automated research capability rides frontier model progress for free; but the scorer is Sakana's own Automated Reviewer grading its own system. Take half of it.
5. Companies and money: the timelines are public, the valuations are feverish
The big labs have moved "automated AI researcher" from vision to roadmap. OpenAI is the most aggressive: on October 28, 2025, Altman and chief scientist Jakub Pachocki publicly targeted an intern-level research assistant by September 2026 and a fully automated "legitimate AI researcher" — capable of autonomously delivering large research projects — by 2028; Pachocki added that superintelligence may be less than a decade away. DeepMind's Hassabis is far more conservative: his Davos framing in January 2026 was 5–10 years to AGI, 50% odds within the decade, with one or two breakthroughs still missing (continual learning, long-term memory, reasoning and planning). The two leading labs' timelines for the same milestone differ by roughly 2x — that spread is the industry's most honest uncertainty gauge.
Among startups, the money has already decided. Sakana AI: a November 2025 Series B of about $135M at a $2.65B post-money — up 1.8x from the $1.5B Series A a year earlier. The investor list rewards a second look: MUFG, Khosla, Macquarie, NEA, Lux — and In-Q-Tel, the CIA-linked strategic investor. Notably, the Series B story is Japanese sovereign AI (Japanese-language models, defense, intelligence, manufacturing); the AI Scientist is the marquee, not the revenue. Wilder still is Periodic Labs: founded by former OpenAI VP of Research Liam Fedus and former DeepMind scientist Ekin Dogus Cubuk, building autonomous robotic laboratories — running physics and chemistry experiments to generate training data for AI, with targets like higher-temperature superconductors. Valued at $1.3B at founding in September 2025; by May 2026, raising at least $500M at $7.5B (led by ex-a16z partner Anjney Midha's AMP) — nearly 6x in under eight months, having recruited 20+ researchers from Meta, OpenAI, and DeepMind. Note what Periodic fixes: it builds the flywheel on physical experimental data, routing around the synthetic-data inbreeding graveyard that haunts pure-software RSI.
6. Measurement: three benchmarks, one shared conclusion
METR's Time Horizon is the single most important curve in this field: the task length (in human time) a model can complete at 50% success. As of the May 8, 2026 page, all 2026 frontier models are on it, with the newest, Claude Mythos Preview (early), measuring a 50% time horizon of ≥16 hours (95% CI 8.5–55h). But METR itself openly concedes: measurements above 16 hours are unreliable with the current task suite — only 5 of 228 tasks are estimated at ≥16 hours. In other words, the strongest models have punched through the measurement ceiling, and "multi-day to multi-week AI R&D tasks" — the interval where the RSI red lines live — fall exactly where the ruler stops working.
RE-Bench (METR) is the only AI-R&D evaluation with a rigorous human baseline: 7 open-ended ML research engineering environments, grounded in 71 eight-hour attempts by 61 human experts. The structure it revealed still holds: at a 2-hour budget the best AI scores 4x human experts; at 32 hours humans flip it to 2x over AI — AI wins on rapid trial-and-error, loses on long-horizon accumulation. Which is why the time-horizon curve matters: it measures precisely the moat that is shrinking.
The closest thing to an "AI trains AI" benchmark is PostTrainBench (March 2026, ICML 2026): give an agent one H100 and 10 hours to fully autonomously post-train a base model into an instruct model — web access, experiments, data curation all self-directed. Read the result both ways: the best agent (Opus 4.6 + Claude Code) averages 23.2% versus 51.1% for official instruction-tuned models — far behind overall; but in targeted settings it already wins: GPT-5.1 Codex Max post-trained Gemma-3-4B to 89% on BFCL function calling versus the official model's 67%. Ahead in the narrow, behind in the broad — that is the entire truth of "AI automating AI training" as of mid-2026.
7. Red lines: what RSI looks like in safety policy
RSI is the only capability written into every leading lab's safety framework as a standalone risk domain (well — almost every: xAI's framework excludes automated AI R&D entirely, without justification). Anthropic's RSP has the most quantified definition: AI R&D-4 = the ability to fully automate the work of an entry-level, remote-only researcher at Anthropic; AI R&D-5 = the ability to cause dramatic acceleration in the rate of effective scaling, quantified as roughly 1000x scale-up within a year. DeepMind's Frontier Safety Framework sets two ML R&D critical capability levels: an acceleration level (has been used to push AI progress substantially past historical rates) and an automation level (can fully automate any Google AI research team at comparable cost) — and DeepMind is the only company of the twelve with frameworks covering all four recommended risk domains.
The frameworks exist; how good are they? SaferAI scored all twelve companies' frameworks against 65 weighted criteria: Anthropic highest at 34%, Cohere lowest at 8%, median 18%. The more uncomfortable finding: for loss-of-control/self-improvement risks, all twelve companies' "mitigations" are essentially open research directions, not executable measures; Anthropic itself acknowledges that assurance processes for advanced AI R&D capabilities do not yet exist, committing only to develop them. Translation: the red lines are drawn, the rulers to detect crossing exist, but what to do after a crossing — nobody knows. Academia's response is to push measurement into the real world: a GovAI paper from March 2026 proposes 14 indicators tracking the actual degree of AI R&D automation — capital share of AI R&D spending, researcher time allocation, counts of AI subversion incidents — shifting from "can the model" to "how fast is the world being changed."
8. Will it explode? The debate compressed into three numbers
Start with prediction tracking. AI 2027 — the famous report that put month-level dates on RSI timelines — now has a community tracker auditing it line by line: 16 of 31 core predictions confirmed, with reality running at about 70% of the predicted pace. Right direction, slower clock: the qualitative calls (coding agents delivering real value, AI companies pivoting to AI-for-AI-research) all hit; the RE-Bench 1.3 target (top-human-expert level) is marked Behind — "too aggressive," says the tracker; the AI R&D progress multiplier of 1.5x (early 2026) is On Track, 2x (late 2026) Emerging. Even the authors' own beliefs have whipsawed: the superhuman-coder median went from 2027 (April 2025) out to late 2029/early 2032 (January 2026), then snapped back ~1.5 years earlier in April 2026 — attributed to Opus 4.6 and METR's doubling period shrinking from 5.5 to 4 months. Forecasters chasing the curve up and down is probably the best evidence that nobody really knows.
Then the theory. Whether a software-only intelligence explosion is possible reduces most cleanly to one economic parameter: σ, the elasticity of substitution between compute and cognitive labor in the AI research production function. σ>1 (substitutes): infinitely copyable AI researchers can fill the compute gap — explosion plausible. σ<1 (complements): any number of AI researchers gets bottlenecked by experiment compute — the loop fizzles. The only empirical attempt so far produced two estimates that contradict each other: a baseline model at σ=2.583 (pro-explosion) and a frontier-experiments model at σ=-0.103 (strong complements, anti) — from data covering just 4 labs and 27 firm-years, which the authors themselves say to read with caution. Practitioner intuition mostly sides with complements: Interconnects' Nathan Lambert calls an AI 2027-style fast takeoff "very unlikely" — lots of progress, no visible acceleration.
Finally, the field study. An interview study of 25 frontier researchers (DeepMind, OpenAI, Anthropic, Meta, Berkeley, Princeton, Stanford; conducted August–September 2025) is the most honest collective snapshot available: 20 of 25 named automating AI research one of the most severe and urgent AI risks; among transcripts discussing constraints, compute was cited most (11 of 16), then data (5); and optimists and skeptics shared the same distinction — AI's research execution and research ideation are different things, with ideation widely expected to be harder. That distinction matches the 1,250-paper survey's conclusion exactly: research execution is verifiable (code runs, benchmarks score) and is being automated at scale; research direction-setting is the paradigm case of a non-verifiable task — and it is where autonomous research loops systematically fail, and where humans remain.
9. Closing
Rewrite the sixty-year-old philosophy problem against mid-2026 evidence and it reads like this: RSI is real, but only where the evaluator can hold the line. Code and math, blessed with cheap reliable verifiers, have industrialized self-improvement; research execution is being swallowed; research taste — judging what is worth doing — shows no automation path yet. Whether it explodes rests on two parameters nobody can reliably estimate: whether compute and intelligence are substitutes or complements, and whether unverifiable tasks can be outflanked by verifiable ones. The industry's own vote is split: safety teams write RSI as red line number one, growth teams write it into the 2028 roadmap, and capital prices the latter. The only certainty is about measurement itself: with the strongest models punching through the benchmarks' reliable ceiling, our answer to "how far from the red line?" is getting less precise, not more.
(Methods note: research for this piece was gathered by 113 parallel agents across 30 sources; of 149 candidate claims, 25 went through three-vote adversarial verification and 23 survived. Every key number in the text comes from a surviving claim or carries an explicit qualifier. Main sources: arXiv 2607.13104, 2607.07663, 2505.22954, 2506.10943, 2603.08640, 2411.15114, 2603.03338, 2603.03992; metr.org; sakana.ai; Nature s41586-026-10265-5; anthropic.com RSP; TechCrunch, Forbes, Fortune; ai2027-tracker.com; interconnects.ai.)
← Back to all articles
1. First, the definition: the field only got formal language in 2026
For years the definition of RSI belonged to LessWrong and science fiction. In July 2026 that changed: a survey from Schmidhuber's group (arXiv 2607.13104, posted July 14) formalized self-improvement for the first time as a clean concept — the self-induced update operator: a system that obtains and commits updates to itself. There are only two things to update, and the entire technical map of the field hides in this split: model parameters (θ→θ+1 — self-generated training data, self-evaluative feedback, exploratory experience) or the scaffold (Σ→Σ+1 — prompt optimization, memory, tool creation, full rewrite of the agent's own code). The flagship of the former is MIT's SEAL; of the latter, Sakana's Darwin Gödel Machine — both below.
The same month, another survey (arXiv 2607.07663, July 8) did something more concrete: it read and classified roughly 1,250 relevant papers from 2024–2026. Its conclusion is the field's true status: "the literature's mass sits where a human still audits the loop." The distribution: 393 papers on deployment-time self-refinement, 340 on training bootstraps, 318 on self-evaluation — versus only 139 touching automated research and 60 touching safety. About 74% of the papers are from 2026 — the field is exploding, but what's exploding is "weak RSI" (human-in-the-loop, bounded, evaluable self-improvement). Genuinely closed-loop self-modification remains rare. That is the baseline for reading through all the hype: today's RSI is a field full of human guardrails, and "strong RSI" (unbounded, closed-loop, self-directed) still exists only as the hypothetical adversary in safety policies.
2. Two technical pathways, each with one working flagship
The scaffold pathway's flagship is the Darwin Gödel Machine (DGM, Sakana AI with Jeff Clune's UBC group, ICLR 2026 poster). The name comes from Schmidhuber's 2003 Gödel Machine — the theoretical construct requiring a formal proof of benefit before any self-modification, which nobody built in sixty years because such proofs are impossible in practice. DGM's core move is swapping proof for evidence: the agent iteratively modifies its own code, validates each change empirically on coding benchmarks, and maintains an archive of agents for Darwinian open-ended exploration — keeping the whole branching tree rather than just the best variant, mutating from any node. Results: self-modified code took SWE-bench from 20.0% to 50.0% and Polyglot from 14.2% to 30.7% — more than doubling — and the improvements it discovered on its own (better code-editing tools, long-context management, a peer-review mechanism) are exactly what human engineers would build, except nobody told it to. Two caveats: it modifies the agent scaffold while the underlying model weights stay frozen, and it used a SWE-bench subset. On the same pathway, the earlier STOP (recursive scaffold self-improvement with a frozen LLM) and ADAS (automated design of agentic systems) were the academic prelude; DGM is the current synthesis.
The parameter pathway's flagship is MIT's SEAL (NeurIPS 2025). The model generates its own finetuning data and update directives ("self-edits"), which become persistent weight updates via SFT — and crucially, the generation policy itself is trained with RL, where the reward is the downstream performance of the updated model. The model is learning how to teach itself. The signature number: on knowledge incorporation, Qwen2.5-7B using self-generated data lifted no-context SQuAD from 33.5% to 47.0%, beating synthetic data generated by GPT-4.1 (46.3%) — a 7B model's RL-optimized self-curriculum outperforming a frontier model's curriculum written for it. The 0.7-point margin is small; the directional meaning is not: the optimal supplier for the data flywheel may be the model itself.
The third, evolutionary route (FunSearch, AlphaEvolve: LLM as mutation operator, evolving codebases) was covered in detail in the sister piece; one positioning note here: it is the only route in production — AlphaEvolve recovers 0.7% of Google's global compute and sped up a key Gemini training kernel by 23%, running in production for over a year. Assemble the three routes and you get the state of play: the evolutionary route ships, the scaffold route self-bootstraps best, the parameter route comes closest to the literal meaning of "changing oneself" — and all three stay inside the safety zone where the evaluator has the final word.
3. The evaluator is the ceiling of the whole game
The sharpest insight in that 1,250-paper survey deserves its own section: demonstrated self-improvement strength decays strictly along a verification hierarchy. Where there's a formal verifier (mathematical proof, a compiler), iteration can run indefinitely. Where there's only execution feedback (does the code run), improvement is bounded. Where the evaluator is a learned judge, the judge degrades across iterations. Where the signal is the model's own self-assessment, it is most easily gamed. In one sentence: an RSI loop's ceiling is exactly the quality of its evaluator.
This is not an abstract worry. The self-rewarding route (generator and evaluator sharing weights) has a structural failure mode confirmed independently by multiple papers: the loop systematically over-rewards high-confidence mistakes — the errors the model is most sure about get reinforced first, and this requires no adversary; it is the default. Work like CREAM exists specifically to patch this (consistency regularization), and the existence of the patch literature is itself proof of the disease. Put this next to the verification hierarchy and you understand why every working RSI system today lives in code and math: those are the only two domains with cheap, reliable, unbribable evaluators.
4. The AI Scientist reaches Nature: a milestone, and its asterisks
The automated-research-agent line got its institutional certification on March 26, 2026: Sakana AI's The AI Scientist (with UBC, Vector Institute, and Oxford) was published as a peer-reviewed paper in Nature (Lu et al., Nature 651:914–919). Read it precisely: what reached Nature is the methods paper describing the system — human authors, human reviewers — not an AI-written paper passing Nature review. The AI-written papers have a separate scoreboard: AI Scientist-v2 produced the first fully AI-generated paper to pass human peer review, scoring an average 6.33 (6/7/6) in double-blind review at the ICLR 2025 ICBINB workshop — above the human acceptance threshold, better than 55% of human submissions. The asterisks, reported honestly: workshop bars are far lower than main conferences; the paper was withdrawn as pre-planned; it contained citation errors; Sakana itself concedes none of its three submissions met their internal main-conference standard. External critics go further — one analysis claims roughly 57% of v2's generated papers contain fabricated data. Sakana also offered a self-serving observation: as the underlying foundation models improve, generated-paper quality rises correspondingly — what they call "a scaling law for AI science." If true, automated research capability rides frontier model progress for free; but the scorer is Sakana's own Automated Reviewer grading its own system. Take half of it.
5. Companies and money: the timelines are public, the valuations are feverish
The big labs have moved "automated AI researcher" from vision to roadmap. OpenAI is the most aggressive: on October 28, 2025, Altman and chief scientist Jakub Pachocki publicly targeted an intern-level research assistant by September 2026 and a fully automated "legitimate AI researcher" — capable of autonomously delivering large research projects — by 2028; Pachocki added that superintelligence may be less than a decade away. DeepMind's Hassabis is far more conservative: his Davos framing in January 2026 was 5–10 years to AGI, 50% odds within the decade, with one or two breakthroughs still missing (continual learning, long-term memory, reasoning and planning). The two leading labs' timelines for the same milestone differ by roughly 2x — that spread is the industry's most honest uncertainty gauge.
Among startups, the money has already decided. Sakana AI: a November 2025 Series B of about $135M at a $2.65B post-money — up 1.8x from the $1.5B Series A a year earlier. The investor list rewards a second look: MUFG, Khosla, Macquarie, NEA, Lux — and In-Q-Tel, the CIA-linked strategic investor. Notably, the Series B story is Japanese sovereign AI (Japanese-language models, defense, intelligence, manufacturing); the AI Scientist is the marquee, not the revenue. Wilder still is Periodic Labs: founded by former OpenAI VP of Research Liam Fedus and former DeepMind scientist Ekin Dogus Cubuk, building autonomous robotic laboratories — running physics and chemistry experiments to generate training data for AI, with targets like higher-temperature superconductors. Valued at $1.3B at founding in September 2025; by May 2026, raising at least $500M at $7.5B (led by ex-a16z partner Anjney Midha's AMP) — nearly 6x in under eight months, having recruited 20+ researchers from Meta, OpenAI, and DeepMind. Note what Periodic fixes: it builds the flywheel on physical experimental data, routing around the synthetic-data inbreeding graveyard that haunts pure-software RSI.
6. Measurement: three benchmarks, one shared conclusion
METR's Time Horizon is the single most important curve in this field: the task length (in human time) a model can complete at 50% success. As of the May 8, 2026 page, all 2026 frontier models are on it, with the newest, Claude Mythos Preview (early), measuring a 50% time horizon of ≥16 hours (95% CI 8.5–55h). But METR itself openly concedes: measurements above 16 hours are unreliable with the current task suite — only 5 of 228 tasks are estimated at ≥16 hours. In other words, the strongest models have punched through the measurement ceiling, and "multi-day to multi-week AI R&D tasks" — the interval where the RSI red lines live — fall exactly where the ruler stops working.
RE-Bench (METR) is the only AI-R&D evaluation with a rigorous human baseline: 7 open-ended ML research engineering environments, grounded in 71 eight-hour attempts by 61 human experts. The structure it revealed still holds: at a 2-hour budget the best AI scores 4x human experts; at 32 hours humans flip it to 2x over AI — AI wins on rapid trial-and-error, loses on long-horizon accumulation. Which is why the time-horizon curve matters: it measures precisely the moat that is shrinking.
The closest thing to an "AI trains AI" benchmark is PostTrainBench (March 2026, ICML 2026): give an agent one H100 and 10 hours to fully autonomously post-train a base model into an instruct model — web access, experiments, data curation all self-directed. Read the result both ways: the best agent (Opus 4.6 + Claude Code) averages 23.2% versus 51.1% for official instruction-tuned models — far behind overall; but in targeted settings it already wins: GPT-5.1 Codex Max post-trained Gemma-3-4B to 89% on BFCL function calling versus the official model's 67%. Ahead in the narrow, behind in the broad — that is the entire truth of "AI automating AI training" as of mid-2026.
7. Red lines: what RSI looks like in safety policy
RSI is the only capability written into every leading lab's safety framework as a standalone risk domain (well — almost every: xAI's framework excludes automated AI R&D entirely, without justification). Anthropic's RSP has the most quantified definition: AI R&D-4 = the ability to fully automate the work of an entry-level, remote-only researcher at Anthropic; AI R&D-5 = the ability to cause dramatic acceleration in the rate of effective scaling, quantified as roughly 1000x scale-up within a year. DeepMind's Frontier Safety Framework sets two ML R&D critical capability levels: an acceleration level (has been used to push AI progress substantially past historical rates) and an automation level (can fully automate any Google AI research team at comparable cost) — and DeepMind is the only company of the twelve with frameworks covering all four recommended risk domains.
The frameworks exist; how good are they? SaferAI scored all twelve companies' frameworks against 65 weighted criteria: Anthropic highest at 34%, Cohere lowest at 8%, median 18%. The more uncomfortable finding: for loss-of-control/self-improvement risks, all twelve companies' "mitigations" are essentially open research directions, not executable measures; Anthropic itself acknowledges that assurance processes for advanced AI R&D capabilities do not yet exist, committing only to develop them. Translation: the red lines are drawn, the rulers to detect crossing exist, but what to do after a crossing — nobody knows. Academia's response is to push measurement into the real world: a GovAI paper from March 2026 proposes 14 indicators tracking the actual degree of AI R&D automation — capital share of AI R&D spending, researcher time allocation, counts of AI subversion incidents — shifting from "can the model" to "how fast is the world being changed."
8. Will it explode? The debate compressed into three numbers
Start with prediction tracking. AI 2027 — the famous report that put month-level dates on RSI timelines — now has a community tracker auditing it line by line: 16 of 31 core predictions confirmed, with reality running at about 70% of the predicted pace. Right direction, slower clock: the qualitative calls (coding agents delivering real value, AI companies pivoting to AI-for-AI-research) all hit; the RE-Bench 1.3 target (top-human-expert level) is marked Behind — "too aggressive," says the tracker; the AI R&D progress multiplier of 1.5x (early 2026) is On Track, 2x (late 2026) Emerging. Even the authors' own beliefs have whipsawed: the superhuman-coder median went from 2027 (April 2025) out to late 2029/early 2032 (January 2026), then snapped back ~1.5 years earlier in April 2026 — attributed to Opus 4.6 and METR's doubling period shrinking from 5.5 to 4 months. Forecasters chasing the curve up and down is probably the best evidence that nobody really knows.
Then the theory. Whether a software-only intelligence explosion is possible reduces most cleanly to one economic parameter: σ, the elasticity of substitution between compute and cognitive labor in the AI research production function. σ>1 (substitutes): infinitely copyable AI researchers can fill the compute gap — explosion plausible. σ<1 (complements): any number of AI researchers gets bottlenecked by experiment compute — the loop fizzles. The only empirical attempt so far produced two estimates that contradict each other: a baseline model at σ=2.583 (pro-explosion) and a frontier-experiments model at σ=-0.103 (strong complements, anti) — from data covering just 4 labs and 27 firm-years, which the authors themselves say to read with caution. Practitioner intuition mostly sides with complements: Interconnects' Nathan Lambert calls an AI 2027-style fast takeoff "very unlikely" — lots of progress, no visible acceleration.
Finally, the field study. An interview study of 25 frontier researchers (DeepMind, OpenAI, Anthropic, Meta, Berkeley, Princeton, Stanford; conducted August–September 2025) is the most honest collective snapshot available: 20 of 25 named automating AI research one of the most severe and urgent AI risks; among transcripts discussing constraints, compute was cited most (11 of 16), then data (5); and optimists and skeptics shared the same distinction — AI's research execution and research ideation are different things, with ideation widely expected to be harder. That distinction matches the 1,250-paper survey's conclusion exactly: research execution is verifiable (code runs, benchmarks score) and is being automated at scale; research direction-setting is the paradigm case of a non-verifiable task — and it is where autonomous research loops systematically fail, and where humans remain.
9. Closing
Rewrite the sixty-year-old philosophy problem against mid-2026 evidence and it reads like this: RSI is real, but only where the evaluator can hold the line. Code and math, blessed with cheap reliable verifiers, have industrialized self-improvement; research execution is being swallowed; research taste — judging what is worth doing — shows no automation path yet. Whether it explodes rests on two parameters nobody can reliably estimate: whether compute and intelligence are substitutes or complements, and whether unverifiable tasks can be outflanked by verifiable ones. The industry's own vote is split: safety teams write RSI as red line number one, growth teams write it into the 2028 roadmap, and capital prices the latter. The only certainty is about measurement itself: with the strongest models punching through the benchmarks' reliable ceiling, our answer to "how far from the red line?" is getting less precise, not more.
(Methods note: research for this piece was gathered by 113 parallel agents across 30 sources; of 149 candidate claims, 25 went through three-vote adversarial verification and 23 survived. Every key number in the text comes from a surviving claim or carries an explicit qualifier. Main sources: arXiv 2607.13104, 2607.07663, 2505.22954, 2506.10943, 2603.08640, 2411.15114, 2603.03338, 2603.03992; metr.org; sakana.ai; Nature s41586-026-10265-5; anthropic.com RSP; TechCrunch, Forbes, Fortune; ai2027-tracker.com; interconnects.ai.)
Comments