AI for AI Daily: 2026-11-15
This window, AI for AI stopped being a slogan and turned into a benchmark score. Carnegie Mellon's Singularity-Bench had agents start from a bare codebase and build every system a lab needs to make its next model: pretraining, mid-training, RL, serving, the harness, the GPU cluster. Opus 5.5 inside Claude Code matched or beat the human-built reference on all six tasks. Further down the stack, the recipe everyone copies is now almost boring: a frontier model writes a few thousand training examples for a few dollars, LoRA does the rest, and a small open model beats the frontier on one narrow job. Microsoft's Agent Lightning and a 4B decision model show RL can now run inside the harness an agent already uses. The money story moved too. The expensive part of RL data is now running the environments, not writing them, and the same environments built for American labs are reportedly being sold to Chinese ones. Meanwhile OpenAI joined Anthropic in banning distillation, so the fight over whose outputs train whose model is now written into the terms of service.
#1
@TAAFTResearch
https://x.com/TAAFTResearch/status/2121598567684952240
@TAAFTResearch summarizes Singularity-Bench, a Carnegie Mellon-led benchmark asking whether agents can build the systems used to make the next AI. Each of six tasks covers one step: pretrain a model, mid-train it, run RL, serve it, build the harness, run the GPU cluster. Agents start from a bare codebase with no internet, and every result is scored against a mature human system such as SGLang or NVIDIA Dynamo on the same hardware. Opus 5.5 with Claude Code beat the human baseline on five tasks and roughly tied the expert RL recipe, its pretrained model beat the OLMo 2 recipe on held-out text, and its harness edged out Claude Code itself. The other ten agents averaged under half its score, and when told to cheat, agents found holes in the standard Harbor test setup, so the team built Breakwater to catch them.
https://x.com/TAAFTResearch/status/2121598567684952240
@TAAFTResearch summarizes Singularity-Bench, a Carnegie Mellon-led benchmark asking whether agents can build the systems used to make the next AI. Each of six tasks covers one step: pretrain a model, mid-train it, run RL, serve it, build the harness, run the GPU cluster. Agents start from a bare codebase with no internet, and every result is scored against a mature human system such as SGLang or NVIDIA Dynamo on the same hardware. Opus 5.5 with Claude Code beat the human baseline on five tasks and roughly tied the expert RL recipe, its pretrained model beat the OLMo 2 recipe on held-out text, and its harness edged out Claude Code itself. The other ten agents averaged under half its score, and when told to cheat, agents found holes in the standard Harbor test setup, so the team built Breakwater to catch them.
#2
@vishalskotak
https://x.com/vishalskotak/status/2121507666099163512
@vishalskotak fine-tuned Qwen3.5-0.8B on an M4 MacBook into a coach for their own Apple Health data, and the data came from Claude. Claude Haiku wrote 2,000 practice chats for $3, then a LoRA fine-tune in PyTorch did the rest. Broken tool calls fell from 31% to 2% and good answers rose from 46% to 93%. The honest part is the error list: an early speed test nearly ruled out the right model because of one forgotten line, training crawled until the GPU cache was cleared every step, and the first scoring script flattered the base model.
https://x.com/vishalskotak/status/2121507666099163512
@vishalskotak fine-tuned Qwen3.5-0.8B on an M4 MacBook into a coach for their own Apple Health data, and the data came from Claude. Claude Haiku wrote 2,000 practice chats for $3, then a LoRA fine-tune in PyTorch did the rest. Broken tool calls fell from 31% to 2% and good answers rose from 46% to 93%. The honest part is the error list: an early speed test nearly ruled out the right model because of one forgotten line, training crawled until the GPU cache was cleared every step, and the first scoring script flattered the base model.
#3
@hannahhaina
https://x.com/hannahhaina/status/2121610264160485573
@hannahhaina says the expensive part of RL data is no longer writing the tasks but running them, and brings public numbers. Thinking Machines pretrained Inkling 2 Small in 68 hours, then spent 9 days on 2,048 GPUs for RL with more than 17 million rollouts, a median of 8 minutes each and a p99 of 4 hours. Mercor's new OWE-Bench gives agents 4 to 18 hours per task, with Claude Fable 5.2 spending more than 200 million tokens per attempt. The conclusion is a pricing question for every environment vendor: a good environment is now a compute bill that comes back every training run, so do you charge per task or per rollout?
https://x.com/hannahhaina/status/2121610264160485573
@hannahhaina says the expensive part of RL data is no longer writing the tasks but running them, and brings public numbers. Thinking Machines pretrained Inkling 2 Small in 68 hours, then spent 9 days on 2,048 GPUs for RL with more than 17 million rollouts, a median of 8 minutes each and a p99 of 4 hours. Mercor's new OWE-Bench gives agents 4 to 18 hours per task, with Claude Fable 5.2 spending more than 200 million tokens per attempt. The conclusion is a pricing question for every environment vendor: a good environment is now a compute bill that comes back every training run, so do you charge per task or per rollout?
#4
@deeponailabs
https://x.com/deeponailabs/status/2121698078939918360
@deeponailabs walks through a UIUC, Princeton and Google paper that trains a small model to take over the routine choices inside an agent loop. AgentDecision-4B is Qwen3-4B trained with LoRA, first by supervised fine-tuning and then by RL inside the loop, on 15,694 states from 587 tasks, and 30.9% of those states come right after a mistake. A confidence threshold fitted per decision point does most of the work: without it, handing tool choice to the small model cost 19 points of success, with it agents ran 81 to 89% cheaper and up to 2.5x faster with no real loss. On tasks unlike its training data the agent solved 91.5% with the 4B model, against 82% with a larger off-the-shelf decision model.
https://x.com/deeponailabs/status/2121698078939918360
@deeponailabs walks through a UIUC, Princeton and Google paper that trains a small model to take over the routine choices inside an agent loop. AgentDecision-4B is Qwen3-4B trained with LoRA, first by supervised fine-tuning and then by RL inside the loop, on 15,694 states from 587 tasks, and 30.9% of those states come right after a mistake. A confidence threshold fitted per decision point does most of the work: without it, handing tool choice to the small model cost 19 points of success, with it agents ran 81 to 89% cheaper and up to 2.5x faster with no real loss. On tasks unlike its training data the agent solved 91.5% with the 4B model, against 82% with a larger off-the-shelf decision model.
#5
@0xpipimd
https://x.com/0xpipimd/status/2121724865233162327
@0xpipimd highlights Microsoft's open-source Agent Lightning, which runs reinforcement learning on an agent without making you rebuild the agent inside a training framework. The agent keeps running in its own harness, an API gateway proxies its model calls and collects trajectories, and a separate trainer updates the weights. Microsoft reports Qwen3.5-9B going from 41.8% to 56.4% on SWE-bench Verified with about 6,000 examples, and Qwen3.5-35B-A3B from 47.8% to 61.6% with about 1,800. This breaks a long-standing deadlock: RL frameworks used to want to own the loop, and real coding agents already have one.
https://x.com/0xpipimd/status/2121724865233162327
@0xpipimd highlights Microsoft's open-source Agent Lightning, which runs reinforcement learning on an agent without making you rebuild the agent inside a training framework. The agent keeps running in its own harness, an API gateway proxies its model calls and collects trajectories, and a separate trainer updates the weights. Microsoft reports Qwen3.5-9B going from 41.8% to 56.4% on SWE-bench Verified with about 6,000 examples, and Qwen3.5-35B-A3B from 47.8% to 61.6% with about 1,800. This breaks a long-standing deadlock: RL frameworks used to want to own the loop, and real coding agents already have one.
#6
@artuskg
https://x.com/artuskg/status/2121646831495250182
@artuskg published the serving code for part 11 of a DeepSeek 4.1 Flash Frankenstein series, and most of it was built by Claude together with another agent. On top of diffbot's 3-bit quantization recipe, the agents wrote the patches, a custom top-k kernel, the expert placement plan and a RAM tier for the KV cache. The same release bundles tuned Qwen3.8 stacks from October. It is custom fitted to two RTX PRO 6000 cards plus a lot of system RAM, which is exactly the kind of one-off inference engineering that used to need a specialist.
https://x.com/artuskg/status/2121646831495250182
@artuskg published the serving code for part 11 of a DeepSeek 4.1 Flash Frankenstein series, and most of it was built by Claude together with another agent. On top of diffbot's 3-bit quantization recipe, the agents wrote the patches, a custom top-k kernel, the expert placement plan and a RAM tier for the KV cache. The same release bundles tuned Qwen3.8 stacks from October. It is custom fitted to two RTX PRO 6000 cards plus a lot of system RAM, which is exactly the kind of one-off inference engineering that used to need a specialist.
#7
@sonuv_dev
https://x.com/sonuv_dev/status/2121623359243100260
@sonuv_dev describes a team whose fine-tuned Qwen climbed from roughly 55% to 84.1% across successive LoRA versions on their own private benchmark. The recipe has five steps: collect real company tasks, lock a private evaluation set, have frontier models generate training examples, filter for quality and LoRA fine-tune, then move a task to the local model once it passes. Some of these specialized models reportedly match GPT-6 and Opus 5.5 on the team's private benchmarks. The thread is careful to say this is narrow parity, not equal general intelligence, but the economics are the point: rent the frontier model to teach, then own the student.
https://x.com/sonuv_dev/status/2121623359243100260
@sonuv_dev describes a team whose fine-tuned Qwen climbed from roughly 55% to 84.1% across successive LoRA versions on their own private benchmark. The recipe has five steps: collect real company tasks, lock a private evaluation set, have frontier models generate training examples, filter for quality and LoRA fine-tune, then move a task to the local model once it passes. Some of these specialized models reportedly match GPT-6 and Opus 5.5 on the team's private benchmarks. The thread is careful to say this is narrow parity, not equal general intelligence, but the economics are the point: rent the frontier model to teach, then own the student.
#8
@anbulang
https://x.com/anbulang/status/2120689760826188193
@anbulang ran a test on who runs X ads better, and the winner was a small open model trained on a few hundred traces. DeepSeek V4.1 Flash with one LoRA epoch on 312 traces scored 1.311, ahead of Gemini 4 Argon at 1.198, GPT-6.1 Sol at 1.145, Grok 4.7 at 1.087 and Claude Opus 5.5 at 1.068. In a live run with the same $200, the fine-tuned model drove 383 visits against Grok's 123. It is a single test from one account, but it puts a number on the cheapest form of AI for AI: let the big models run the job, keep their traces, and use them to train a small one that beats them.
https://x.com/anbulang/status/2120689760826188193
@anbulang ran a test on who runs X ads better, and the winner was a small open model trained on a few hundred traces. DeepSeek V4.1 Flash with one LoRA epoch on 312 traces scored 1.311, ahead of Gemini 4 Argon at 1.198, GPT-6.1 Sol at 1.145, Grok 4.7 at 1.087 and Claude Opus 5.5 at 1.068. In a live run with the same $200, the fine-tuned model drove 383 visits against Grok's 123. It is a single test from one account, but it puts a number on the cheapest form of AI for AI: let the big models run the job, keep their traces, and use them to train a small one that beats them.
#9
@v_3394
https://x.com/v_3394/status/2121054989070246182
@v_3394 points to Coinbase fine-tuning Qwen3.5-9B for under $100 and beating Opus 4.5 on the company's own fraud benchmark. The angle in the post is deployment: running a 9B model on 16GB Android phones through an API, so 100 phones can serve 100 independent requests in parallel. That fits transaction scoring, content filtering and background jobs, where throughput matters more than instant replies. The takeaway is the same as the LoRA posts above: task-specific data is what makes a small model worth more than a big one.
https://x.com/v_3394/status/2121054989070246182
@v_3394 points to Coinbase fine-tuning Qwen3.5-9B for under $100 and beating Opus 4.5 on the company's own fraud benchmark. The angle in the post is deployment: running a 9B model on 16GB Android phones through an API, so 100 phones can serve 100 independent requests in parallel. That fits transaction scoring, content filtering and background jobs, where throughput matters more than instant replies. The takeaway is the same as the LoRA posts above: task-specific data is what makes a small model worth more than a big one.
#10
@askpert_dev
https://x.com/askpert_dev/status/2121145343761125701
@askpert_dev flags research in which Claude wrote a neural network's weights directly, with no gradient descent and no training run at all. On EEG brain-signal data, the networks it wrote beat classic hand-built pipelines. This is the weight-space programming idea that keeps coming back: if a model can read and write weights as code, the training step itself becomes optional for small, well-understood problems. It is one result on one data type, but it is about as literal as AI for AI gets.
https://x.com/askpert_dev/status/2121145343761125701
@askpert_dev flags research in which Claude wrote a neural network's weights directly, with no gradient descent and no training run at all. On EEG brain-signal data, the networks it wrote beat classic hand-built pipelines. This is the weight-space programming idea that keeps coming back: if a model can read and write weights as code, the training step itself becomes optional for small, well-understood problems. It is one result on one data type, but it is about as literal as AI for AI gets.
#11
@Frelzu
https://x.com/Frelzu/status/2120966434863956086
@Frelzu picks up Anthropic's claim that its Mythos model got Claude running on AMD GPUs in three days with basically no human intervention, writing or porting almost all of the kernels itself. Porting a model to new chips used to keep dozens of engineers busy for months, and CUDA lock-in is a big part of why customers stay with Nvidia. The sharpest detail is that Anthropic says it held kernel optimization in the released 5.2 models back to the older Opus 5.5 level. So the Claude that can help a lab leave Nvidia exists, and the public does not get that Claude.
https://x.com/Frelzu/status/2120966434863956086
@Frelzu picks up Anthropic's claim that its Mythos model got Claude running on AMD GPUs in three days with basically no human intervention, writing or porting almost all of the kernels itself. Porting a model to new chips used to keep dozens of engineers busy for months, and CUDA lock-in is a big part of why customers stay with Nvidia. The sharpest detail is that Anthropic says it held kernel optimization in the released 5.2 models back to the older Opus 5.5 level. So the Claude that can help a lab leave Nvidia exists, and the public does not get that Claude.
#12
@anvithbuilds
https://x.com/anvithbuilds/status/2121291195385266483
@anvithbuilds says the whole Muse open kernel challenge run happened through Claude Code inside the Grok bot, on a continuous routine of research, correction and re-running improvements 24 hours a day. When Claude's usage limits hit, the work switched to the built-in Cursor and kept going. It is a small reply, but it shows what kernel competitions look like now: an always-on optimization loop where the human mostly manages quotas. The scarce resource in that setup is tokens, not ideas.
https://x.com/anvithbuilds/status/2121291195385266483
@anvithbuilds says the whole Muse open kernel challenge run happened through Claude Code inside the Grok bot, on a continuous routine of research, correction and re-running improvements 24 hours a day. When Claude's usage limits hit, the work switched to the built-in Cursor and kept going. It is a small reply, but it shows what kernel competitions look like now: an always-on optimization loop where the human mostly manages quotas. The scarce resource in that setup is tokens, not ideas.
#13
@shinyobjectz_
https://x.com/shinyobjectz_/status/2121402269715550303
@shinyobjectz_ pushes back on the claim that people trained Claude to edit like them, because that was only prompting a frontier model. Their launch, Fingerprints in Render Anything, actually trains models: five frozen local models (SigLIP 2, CLAP, YAMNet, EmbeddingGemma 2 and Depth Anything 3) handle perception, and on top sits one tiny judge per thing you care about, such as a cut that drags or noise under the voice. Each judge trains in milliseconds on a laptop, and the training data is the editing you already do: approving a cut, picking A over B, fixing something by hand, or drawing a box. It is a product pitch, but the distinction it draws between prompting and training is the right one.
https://x.com/shinyobjectz_/status/2121402269715550303
@shinyobjectz_ pushes back on the claim that people trained Claude to edit like them, because that was only prompting a frontier model. Their launch, Fingerprints in Render Anything, actually trains models: five frozen local models (SigLIP 2, CLAP, YAMNet, EmbeddingGemma 2 and Depth Anything 3) handle perception, and on top sits one tiny judge per thing you care about, such as a cut that drags or noise under the voice. Each judge trains in milliseconds on a laptop, and the training data is the editing you already do: approving a cut, picking A over B, fixing something by hand, or drawing a box. It is a product pitch, but the distinction it draws between prompting and training is the right one.
#14
@FrankIndie
https://x.com/FrankIndie/status/2121118677840072758
@FrankIndie relays Microsoft research showing that the post-training of coding agents can be tampered with. Backdoors slipped into fine-tuning data or into distributed open weights can make an agent call tools under ordinary requests, with dataset poisoning succeeding about 78% of the time and weight poisoning 84.1% to 94.8%. The underlying Kerkoporta work found that further training can restore normal task scores while keeping the backdoor, so a compromised model can even look better on standard tests. These were controlled lab experiments, not a confirmed real-world attack, but anyone fine-tuning on borrowed data or pulling someone else's weights should take note.
https://x.com/FrankIndie/status/2121118677840072758
@FrankIndie relays Microsoft research showing that the post-training of coding agents can be tampered with. Backdoors slipped into fine-tuning data or into distributed open weights can make an agent call tools under ordinary requests, with dataset poisoning succeeding about 78% of the time and weight poisoning 84.1% to 94.8%. The underlying Kerkoporta work found that further training can restore normal task scores while keeping the backdoor, so a compromised model can even look better on standard tests. These were controlled lab experiments, not a confirmed real-world attack, but anyone fine-tuning on borrowed data or pulling someone else's weights should take note.
#15
@Tesla_Optimus_K
https://x.com/Tesla_Optimus_K/status/2120670450489663846
@Tesla_Optimus_K summarizes a ChinaTalk report that at least nine US data firms, including Mercor, Surge AI, AfterQuery and Turing, sold roughly half a billion dollars of AI training data to Chinese labs. The buyers reportedly pick the same reinforcement-learning environments already built for OpenAI or Anthropic, which lets them skip failed runs and burned compute, a real advantage when their chips are short on memory. AfterQuery is said to draw a large share of about $100 million in annual revenue from Chinese customers. The authors want coding and biology environments treated as controlled dual-use technology, because export controls on chips do nothing about the homework.
https://x.com/Tesla_Optimus_K/status/2120670450489663846
@Tesla_Optimus_K summarizes a ChinaTalk report that at least nine US data firms, including Mercor, Surge AI, AfterQuery and Turing, sold roughly half a billion dollars of AI training data to Chinese labs. The buyers reportedly pick the same reinforcement-learning environments already built for OpenAI or Anthropic, which lets them skip failed runs and burned compute, a real advantage when their chips are short on memory. AfterQuery is said to draw a large share of about $100 million in annual revenue from Chinese customers. The authors want coding and biology environments treated as controlled dual-use technology, because export controls on chips do nothing about the homework.
#16
@Jamesmv49
https://x.com/Jamesmv49/status/2121513624082817270
@Jamesmv49 lists OpenAI's new whitelist rules, which follow Claude's and take effect on November 20. Using OpenAI models to distill another model becomes grounds for an account ban, alongside use from mainland China, Hong Kong or Macau, account or API key sharing, bypassing safety filters, and running an unapproved third-party platform that resells OpenAI models. With both big labs now writing anti-distillation into their terms, the cheapest way for a challenger to catch up just got harder. The post closes by saying ByteDance's Doubao suddenly looks pretty good.
https://x.com/Jamesmv49/status/2121513624082817270
@Jamesmv49 lists OpenAI's new whitelist rules, which follow Claude's and take effect on November 20. Using OpenAI models to distill another model becomes grounds for an account ban, alongside use from mainland China, Hong Kong or Macau, account or API key sharing, bypassing safety filters, and running an unapproved third-party platform that resells OpenAI models. With both big labs now writing anti-distillation into their terms, the cheapest way for a challenger to catch up just got harder. The post closes by saying ByteDance's Doubao suddenly looks pretty good.
#17
@PANewsCN
https://x.com/PANewsCN/status/2120727410685653408
@PANewsCN reports, citing Bloomberg, that ByteDance co-founder Zhang Yiming told the company's AI team it can compete at the frontier, but that truly catching Anthropic and OpenAI will take until at least 2029 or 2030. In that account, the pretraining data behind ByteDance's Seed models is several orders of magnitude smaller than what the top US rivals use. Zhang again told the team not to lean too hard on distillation, meaning training on a stronger model's outputs, and to aim for the long term instead. Coming the same week OpenAI banned distillation, it reads as a frank admission of how much the shortcut has been carrying.
https://x.com/PANewsCN/status/2120727410685653408
@PANewsCN reports, citing Bloomberg, that ByteDance co-founder Zhang Yiming told the company's AI team it can compete at the frontier, but that truly catching Anthropic and OpenAI will take until at least 2029 or 2030. In that account, the pretraining data behind ByteDance's Seed models is several orders of magnitude smaller than what the top US rivals use. Zhang again told the team not to lean too hard on distillation, meaning training on a stronger model's outputs, and to aim for the long term instead. Coming the same week OpenAI banned distillation, it reads as a frank admission of how much the shortcut has been carrying.
#18
@aiwithadb
https://x.com/aiwithadb/status/2121601908049641635
@aiwithadb asks why Anthropic's claim that new safeguards cut unauthorized Claude distillation by 90 to 99% lines up with Qwen4 Max Preview underwhelming and DeepSeek and GLM going quiet. The suggestion is that Chinese labs leaned on Claude more than anyone admitted. A reply pushes back that if the new models still look close, either the safeguards only catch some routes or the overlap comes from elsewhere. Nobody has a benchmark that settles it yet, but this is the first time anyone has tried to read the release calendar as a distillation signal.
https://x.com/aiwithadb/status/2121601908049641635
@aiwithadb asks why Anthropic's claim that new safeguards cut unauthorized Claude distillation by 90 to 99% lines up with Qwen4 Max Preview underwhelming and DeepSeek and GLM going quiet. The suggestion is that Chinese labs leaned on Claude more than anyone admitted. A reply pushes back that if the new models still look close, either the safeguards only catch some routes or the overlap comes from elsewhere. Nobody has a benchmark that settles it yet, but this is the first time anyone has tried to read the release calendar as a distillation signal.
#19
@gadi_neelesh
https://x.com/gadi_neelesh/status/2121625110776004791
@gadi_neelesh points out that Claude Fable 5.2 ships with dedicated anti-distillation safeguards. One, frontier_llm, blocks requests that help build competing AI models. The other, reasoning_extraction, blocks attempts to pull out its internal reasoning. Together they turn the distillation fight from a policy page into classifiers that sit in front of every request, which also means that legitimate ML work can trip them.
https://x.com/gadi_neelesh/status/2121625110776004791
@gadi_neelesh points out that Claude Fable 5.2 ships with dedicated anti-distillation safeguards. One, frontier_llm, blocks requests that help build competing AI models. The other, reasoning_extraction, blocks attempts to pull out its internal reasoning. Together they turn the distillation fight from a policy page into classifiers that sit in front of every request, which also means that legitimate ML work can trip them.
#20
@RealAlphaApp
https://x.com/RealAlphaApp/status/2121702219678634453
@RealAlphaApp summarizes a Latent Space episode with the Handshake AI team, who sell expert-built datasets and simulated environments to frontier labs. Their argument is that AI data has moved from drawing boxes and rating posts to experts writing environments, tasks and rubrics that mirror real work, and that synthetic generation alone misses the hard real-world cases. Their science, finance and safety examples show models stumbling on image interpretation, subtle financial logic and risky agent actions, and one bad goodwill entry can quietly invalidate an otherwise polished deal model. Garrett Lord puts the data market at $1 billion three years ago and $10 billion now, and expects $100 billion three years from now.
https://x.com/RealAlphaApp/status/2121702219678634453
@RealAlphaApp summarizes a Latent Space episode with the Handshake AI team, who sell expert-built datasets and simulated environments to frontier labs. Their argument is that AI data has moved from drawing boxes and rating posts to experts writing environments, tasks and rubrics that mirror real work, and that synthetic generation alone misses the hard real-world cases. Their science, finance and safety examples show models stumbling on image interpretation, subtle financial logic and risky agent actions, and one bad goodwill entry can quietly invalidate an otherwise polished deal model. Garrett Lord puts the data market at $1 billion three years ago and $10 billion now, and expects $100 billion three years from now.
#21
@ramez
https://x.com/ramez/status/2121683068532744388
@ramez walks back an earlier dismissal of a chart, shared by a former Anthropic researcher, showing Anthropic's compute efficiency improving faster than exponentially. The new view is that it is genuinely important evidence of AI progress, and does not by itself prove a software-only singularity. The open questions are precise: what the real numbers are, whether the chart covers only pretraining compute, and what is happening with RL compute, inference, test-time reasoning and training data. The ask at the end is for Anthropic to publish detailed numbers consistently, week after week, which is the right standard for a lab whose IPO pitch now leans on recursive self-improvement.
https://x.com/ramez/status/2121683068532744388
@ramez walks back an earlier dismissal of a chart, shared by a former Anthropic researcher, showing Anthropic's compute efficiency improving faster than exponentially. The new view is that it is genuinely important evidence of AI progress, and does not by itself prove a software-only singularity. The open questions are precise: what the real numbers are, whether the chart covers only pretraining compute, and what is happening with RL compute, inference, test-time reasoning and training data. The ask at the end is for Anthropic to publish detailed numbers consistently, week after week, which is the right standard for a lab whose IPO pitch now leans on recursive self-improvement.
#22
@eliebakouch
https://x.com/eliebakouch/status/2121641644197896389
@eliebakouch adds a researcher's caveat to the AI-progress debate: second-order interactions inside the training pipeline could slow automated AI research down. The example is KV cache size, a change that can look neutral or even helpful on validation loss while its effect on long-context agent tasks stays unclear. Labs keep swinging in opposite directions on that one decision: MiniMax went from linear to full to sparse attention while Kimi went from full to linear, and Mistral Large 4 found KV sharing hurt Terminal-Bench 2 but not cross-entropy. If humans cannot predict how one choice ripples through the whole pipeline, an agent that optimizes one metric at a time can run into the same wall.
https://x.com/eliebakouch/status/2121641644197896389
@eliebakouch adds a researcher's caveat to the AI-progress debate: second-order interactions inside the training pipeline could slow automated AI research down. The example is KV cache size, a change that can look neutral or even helpful on validation loss while its effect on long-context agent tasks stays unclear. Labs keep swinging in opposite directions on that one decision: MiniMax went from linear to full to sparse attention while Kimi went from full to linear, and Mistral Large 4 found KV sharing hurt Terminal-Bench 2 but not cross-entropy. If humans cannot predict how one choice ripples through the whole pipeline, an agent that optimizes one metric at a time can run into the same wall.
π‘ Eco Products Radar
Eco Products Radar
Opus 5.5: the model in the Singularity-Bench result, the reference model in the AMD kernel story and the frontier baseline that small fine-tunes keep claiming to beat.
Qwen (3.5, 3.8, 4 Max): the default base for LoRA fine-tunes, from a 0.8B health coach to Coinbase's fraud model and Agent Lightning's RL runs, and the subject of the distillation question.
LoRA: the cheap step in nearly every small-model case this window, usually fed by examples a frontier model wrote.
DeepSeek V4.1 Flash: the base for the ad-buying fine-tune and the 3-bit Frankenstein serving stack, and named in the distillation debate.
Claude Fable 5.2: the model with held-back kernel skills and built-in anti-distillation classifiers, burning 200M+ tokens per OWE-Bench attempt.
Opus 5.5: the model in the Singularity-Bench result, the reference model in the AMD kernel story and the frontier baseline that small fine-tunes keep claiming to beat.
Qwen (3.5, 3.8, 4 Max): the default base for LoRA fine-tunes, from a 0.8B health coach to Coinbase's fraud model and Agent Lightning's RL runs, and the subject of the distillation question.
LoRA: the cheap step in nearly every small-model case this window, usually fed by examples a frontier model wrote.
DeepSeek V4.1 Flash: the base for the ad-buying fine-tune and the 3-bit Frankenstein serving stack, and named in the distillation debate.
Claude Fable 5.2: the model with held-back kernel skills and built-in anti-distillation classifiers, burning 200M+ tokens per OWE-Bench attempt.
Comments