November 23, 2026ai-for-ai

AI for AI Daily: 2026-11-23

The most useful AI-for-AI work today was about the boring parts: who writes the test, who pays for the data, and whose transcripts end up in the weights. Meta described an agent loop that builds coding benchmarks from merged internal changes and then repairs them until a set of label-free checks passes. OpenAI's own numbers on research agents say their compute share is doubling about every 30 days, with the biggest lift in pretraining and post-training rather than infrastructure. On the small end, someone turned three months of Claude Code sessions into a LoRA for about $6 a pass, and a 1.7B model fine-tuned for one shell language beat Claude Haiku by 79 points. The distillation story moved from technique to policy: OpenAI now wants routers and gateways to ask permission, and Chinese timelines argued over whether the slowdown in open weights is about locked teachers or locked chips. The teacher has a price now, and the cheapest student is the one trained on your own work.
πŸ’‘#1
@omarsar0
https://x.com/omarsar0/status/2123767076745818281
@omarsar0 highlighted a Meta paper called Benchmark Factory: an agent loop that turns merged internal code changes into coding benchmarks for your own codebase. The agent writes the query that selects tasks, runs models of known relative strength, and keeps repairing the benchmark until it passes quality checks that need no per-task human labels. To test the checks, the authors fed them bad benchmarks on purpose: one far too easy and five with different kinds of broken grading were all rejected, and when model identities were scrambled a bad benchmark slipped through at most 1.4% of the time. On configurations never used to build them, the benchmarks ranked Opus 4.7 high effort above low effort (60.6% vs 54.4%) and GPT-5.5 above GPT-5.4 (65.8% vs 60.5%), and 9 of 10 per-team comparisons agreed. Eval design is one of the jobs an AI researcher does, and here it is automated with the validation built in.
πŸ’‘#2
@fleetingbits
https://x.com/fleetingbits/status/2123930436116361387
@fleetingbits pulled the numbers out of OpenAI's latest post on its internal research agents. Agent spend keeps doubling about every 30 days: in September internal deployment used 2% of AI R&D compute, which on trend becomes about 11% by year end. Success rates on AI R&D tasks are climbing about 2 to 3 points a month outside the nearly saturated sub-15-minute bucket, which would mean roughly 64% on 1-2 hour tasks and 46% on 4-8 hour tasks by the end of the year. The detail worth keeping: agents add about 8 workdays per researcher day in pretraining and post-training, against about 3 in scaling and infra, which suggests the lift comes from reading rollouts and tuning data mixes more than from writing code.
πŸ’‘#3
@fngly343149
https://x.com/fngly343149/status/2123656302182412767
@fngly343149 wrote up tinker-ai-skill, a repo that turns your own Claude Code sessions into a LoRA and then a local GGUF model, which picked up 327 stars in 13 days. The author used the method to build Tess-4-27B and Tess-4.5-27B, both open on Hugging Face. The cost table is the useful part: 100 rows of personal transcripts is about 5M tokens and one training pass on Qwen3.6-35B-A3B costs about $6, while 1,000 rows is about 45M tokens and about $55. The pipeline is nine steps and 11 scripts: collect, redact (credentials replaced by value, a random identity per session), split without cutting tool calls apart, train on Tinker, merge and quantize. Your agent transcripts are now training data, and a few hundred dollars buys a local assistant that works the way you do.
πŸ’‘#4
@mcaldwell_dev
https://x.com/mcaldwell_dev/status/2123602081257083058
@mcaldwell_dev fine-tuned Liquid AI's LFM3 1.7B to write Nushell calls and scored it on 231 held-out requests run in a wasm Nushell sandbox. LFM3 with LoRA got 92.6%, gpt-6-luna 22.1% and claude-haiku-5-5 13.0%. There is a WebGPU demo and a Marimo notebook with the whole training process. A niche shell language is exactly where a tiny specialist beats a general model, because the frontier models have barely seen it and the sandbox gives a clean pass or fail.
πŸ’‘#5
@AGTPinsights
https://x.com/AGTPinsights/status/2123885140229439576
@AGTPinsights summarised what is new in OpenAI's post on adversarial distillation beyond the headline 54% to 0.2% drop in reasoning extraction. Reports cite about 16,000 extraction attempts spread across more than 15,000 users, and OpenAI says it closed a pathway that could have exposed another user's encrypted reasoning, without saying whether anyone used it. Its cyber distillation results suggest that hiding reasoning traces limits how much offensive capability transfers to a student. The reaction is split: Ethan Mollick says the end of easy distillation from OpenAI and Anthropic may explain the slowdown in open weights, while critics say the plots show little benefit and that RL environments and compute matter more.
πŸ’‘#6
@sokrypton
https://x.com/sokrypton/status/2123621067444863218
@sokrypton added a follow-up to the protein forensics on Claude: Fable 5.2 also fails on protein-protein interactions when asked to recall a paired multiple sequence alignment. That suggests the model was trained on single UniProt sequences, not on contigs or whole genomes. It is a neat way to read a training set from the outside: if a model can continue one protein but not the partner that sits next to it in a genome, the genome was never in the data.
πŸ’‘#7
@teortaxesTex
https://x.com/teortaxesTex/status/2123695558502051968
@teortaxesTex surfaced a Chinese lab's September listing for a Code Agent Data Engineer, a role that writes no product code. The job is to turn real software engineering, cybersecurity or research scenarios into RL environments, define evaluation criteria and improve the training data so the model writes better code, with tracks for frontend, mobile, backend, 3D rendering, security and game development. Teortaxes notes the lab went hard on hiring game designers around the same time, ties it to the rumour of reasoning uplift from game-playing, and says the lab's 1017 checkpoint is disproportionately good at making games. Building RL environments is becoming a job title of its own.
πŸ’‘#8
@arshshah
https://x.com/arshshah/status/2123593205526499766
@arshshah quoted the AI R&D lines in Anthropic's final prospectus. On page 36 Anthropic says it already treated the AI R&D capability threshold in its Responsible Scaling Policy as met, based on what it expects from future models rather than what it has observed in current ones. On page 56 it says it recently delayed pre-training a new frontier base model by more than a month to harden security against model weight exfiltration. Both are statements the company carries securities liability for, which makes them harder evidence than a blog post: the AI-for-AI threshold is now a disclosed risk factor, and protecting the weights is now on the training schedule.
πŸ’‘#9
@datadogdevs
https://x.com/datadogdevs/status/2123830606710665554
@datadogdevs said Datadog teamed up with Baseten to fine-tune a model on its own production LLM traces. Generation time dropped from 44.7 seconds to 0.89 seconds, about 50 times faster, and the judge pass rate rose from 45% to 93%. The same release adds a pup finetune command so customers can fine-tune open-weight models on Baseten or Fireworks AI. An observability company already holds the traces, so it has the training set for free, and that is the logic behind turning traces into small models.
πŸ’‘#10
@stas_sorokin_
https://x.com/stas_sorokin_/status/2123595434274812310
@stas_sorokin_ described testing Claude Code against hidden HTML attacks: in all 5 runs the agent never mentioned what it had seen. Their fix is a small model, Firewall 1, that sits in front of the agent and caught 60 of 60 hidden HTML attacks at about 65 ms a page, with open weights and no per-call cost. The weak spot is stated plainly: it still flags 28 of 64 clean full pages, so whole pages are the next training run. Training a guard model to cover for a frontier model's blind spot is a very practical kind of AI for AI.
πŸ’‘#11
@atlaslormlabs
https://x.com/atlaslormlabs/status/2123781456380211353
@atlaslormlabs reported that Snorkel raised a model's score on OpenAI's professional work test from 31.9% to 40.7%. The method is binary-gated training: 95% of attempt groups produced a learning signal, against 21% under plain GRPO. That gap is the real story, because GRPO wastes most of its rollouts when every attempt in a group scores the same, and a training method that keeps the signal alive gets more out of the same compute.
πŸ’‘#12
@markfenner
https://x.com/markfenner/status/2123849380763443400
@markfenner noted in a Friday roundup that Hugging Face released adapters tuning Qwen3.8-27B for the Hermes, OpenCode and pi harnesses. With thinking turned off, the Hermes adapter took the model from 44.7% to 58.8% on Terminal-Bench 2.1, on the same base model. Fine-tuning a model to its harness, instead of only tuning the harness to the model, is a 14-point lever that most people running open models have not pulled yet.
πŸ’‘#13
@AliChherawalla
https://x.com/AliChherawalla/status/2123893032017936771
@AliChherawalla reported that a fine-tuned 8B open model now reads how a film shot was made better than Opus 5.5 and Gemini 4. It gets 69% right, runs 2.4 times faster and costs about 30 times less, on your own GPU. Reading camera moves, lenses and lighting off a frame is a narrow visual skill, and like Nushell it is the kind of niche where a small specialist trained on the right data overtakes a frontier generalist.
πŸ’‘#14
@KomalVardhanL
https://x.com/KomalVardhanL/status/2123665199248900212
@KomalVardhanL posted a six-step autoresearch recipe for nanochat. The objective is to minimise validation loss; the agent may change hyperparameters, architecture and optimizer but not the dataset, the core loop or dependencies. Then hit go: one change at a time, measure, log, keep or revert, and review the log in the morning to pick the best run. In the previous part of the series the author says the agents found tweaks they had missed. The CAN and CANNOT lists are the part to copy, because they are what stops an overnight loop from winning by editing the eval.
πŸ’‘#15
@cephaloform
https://x.com/cephaloform/status/2123853125412946062
@cephaloform mentioned a 2B model trained specifically to run FunSearch, which the author calls an ancient autoresearch harness that even works on base models, but with shell tools added. Training a small model for one research harness is the mirror image of the Hugging Face harness adapters: instead of a big general model driving the search loop, a tiny one is shaped to it.
πŸ’‘#16
@EFEE0180492926
https://x.com/EFEE0180492926/status/2123912674849247483
@EFEE0180492926 pointed out that OpenAI's updated usage policy took effect on November 20, after being announced on October 21. Services that run a middle layer letting third parties call OpenAI models directly, such as model routers and AI gateways, now need OpenAI's review and approval; products that simply use the models are not covered. Semafor named OpenRouter as the most visible example, though OpenAI's text names no company. Anti-distillation is no longer only a model-side defence: the resale door is being closed by policy.
πŸ’‘#17
@Andy_whhh
https://x.com/Andy_whhh/status/2123756458022179184
@Andy_whhh amplified a claim from the blogger Da7em that Chinese models went quiet once Anthropic closed the distillation door. The evidence offered is a list of slow releases: Kimi took about 100 days to ship K3.1 Pro, DeepSeek has no Pro-tier flagship yet, Zhipu has gone three months without a new flagship after shipping every two months, and Qwen4-Max preview landed flat. It is correlation, not proof, but it is now the dominant explanation on Chinese AI Twitter for why the open-weight pace slowed.
πŸ’‘#18
@ohouhou717
https://x.com/ohouhou717/status/2123733341631570132
@ohouhou717 pushed back on the idea that every Chinese model is a distillation. DeepSeek put its memory and cost reductions into public papers, and other labs learning from them is the point of open research. The post separates three things that get blurred together: borrowing from public papers, distilling your own large model into small ones, and training on someone else's outputs without permission. It also refuses to invent insider stories in the other direction. That three-way split is the most useful frame in today's distillation argument.
πŸ“‘ Eco Products Radar
Eco Products Radar
Qwen: the base of tinker-ai-skill's personal LoRAs (Qwen3.6-35B-A3B), the model Hugging Face tuned to the Hermes, OpenCode and pi harnesses (Qwen3.8-27B), and Qwen4-Max in the debate over why Chinese releases slowed.
Claude: the source of the Claude Code sessions turned into LoRAs, the agent that kept quiet about hidden HTML attacks, Fable 5.2 in the protein-forensics follow-up, Haiku as the beaten baseline on Nushell, and Opus as the beaten baseline on film shots.
LoRA and small fine-tunes: personal transcripts into Tess-4-27B, LFM3 1.7B for Nushell, an 8B film-shot reader, Datadog's trace-trained model and the Firewall 1 guard model.
nanochat: the target of the six-step overnight autoresearch recipe, and the training codebase in Karpathy's original loop that still anchors how people describe autoresearch.
OpenAI distillation defences: the reasoning-extraction post, the extraction-campaign numbers and the router-and-gateway approval rule.
← Previous
Ideas Radar: 2026-11-23
Next β†’
Nerfed Daily: 2026-11-23
← Back to all articles

Comments

Loading...
>_