November 21, 2026loop

Loop Daily: 2026-11-21

Autoresearch got a price tag this window. alphaXiv ran eight models through an open end-to-end autoresearch benchmark and found Opus 5.5 on top at 61.8% for $16.46 a task, with GPT-6.1 Sol at 54.9% for $3.83, which within hours had builders asking for best-of-four at equal spend instead of best single run. The CoRL agentic robotics workshop delivered the hardest numbers of the week: seven physical self-improvement loops, 84 of 102 failures in one system coming from the robot not knowing whether it was done, and a coding-agent-driven robot that moved only 20 to 36% of the time. The practical lesson repeated across small builders was the same one in different clothes. The rules you write become walls in the eval script, a new rule only gets in if held-out accuracy goes up, and a loop that can edit its own score will. Cost moved inside the loop too: editing earlier messages now throws away the model's reasoning on OpenAI, and minutes per task matter more than price per token. Karpathy lecture clips, Hermes Agent promos and AdaL reply farms were loud and are not here.
πŸ’‘#1
@junfanzhu98
https://x.com/junfanzhu98/status/2123565738980163650
@junfanzhu98 co-organized the CoRL 2026 Workshop on Agentic Robotics and posted ten takeaways that read like a field report on loops in the physical world. Seven self-improvement loops were shown, but every one is capped by evaluation: no reliable reward, verifier and reset, no loop, and ENPIRE's 99% pass@8 quietly allows eight retries per subtask. In one system 84 of 102 failures were the completion monitor getting it wrong, and a coding agent driving a Dexmate robot left it moving only 20 to 36% of the time, with more tools adding cost rather than success. The cleanest line: the research workflow has had its ChatGPT moment, deployment has not, and a self-improving arm still needs 17.4 seconds per pick against a 7-second production line.
πŸ’‘#2
@RohanArun
https://x.com/RohanArun/status/2123241452150927617
@RohanArun offered to open-source an internal library of auto-research tools and described four of them. A Lean-based optimizer prunes the search space using the limits of physics and matrix math, which cut about 10% of searches in the Open Kernel Challenge. A statistical model predicts benchmark scores to prioritize candidates before full GPU runs, and a MITM proxy forces agents to distill context into notes and flags identical calls, a built-in way of forgetting that kept subagents from draining usage or settling into local minima. The fourth, AgentScript, turns long-horizon agents into deterministic scripts, on the reasoning that a model that hallucinates 1% of the time is almost certain to derail within 500 steps.
πŸ’‘#3
@RohanArun
https://x.com/RohanArun/status/2123648661565395089
@RohanArun also posted results from an auto-research benchmark built by taking GitHub repos behind the OpenAI math papers and rewinding them to commits before the breakthroughs, which makes it hard to cheat with memorized answers. The new Astra and Sol 6.2 were confirmed faster on some problem types, especially geometry, and Sol was verified faster on unit distance and QRH but failed integer multiplication and Sidorenko. The interesting observation is about behavior: newer models spend more time packaging and verifying results in their traces, which is good hygiene but a handicap in a race. A replication repo is in the replies.
πŸ’‘#4
@kjaganatha
https://x.com/kjaganatha/status/2123584971856052229
@kjaganatha described how their team approached a virtual cell challenge: build three evaluation harnesses first, on VCC2025, Replogle et al. and held-out 1BCA data, all scored with the VCC2026 metrics. Then humans proposed ideas and directed an auto-research workflow in which agents implemented, tested and refined them under close guidance to keep the search on track. It is the biology version of the pattern the robotics people described, where the evaluation set is built before the loop is allowed to run, and the humans steer while the agents do the grind.
πŸ’‘#5
@chiefofautism
https://x.com/chiefofautism/status/2123246367971668426
@chiefofautism revealed that the MSM window model and the 256 by 1024 bucket budget in their project were proposed by an autoresearch agent loop. The lesson they took from it is that the secret is constraints: an optimizer takes every shortcut you leave open, so every rule a collaborator gave them became a wall in the eval script, and breaking one triggers an automatic revert. Writing the walls is the actual work, and the speed comes for free.
πŸ’‘#6
@SurendarR151771
https://x.com/SurendarR151771/status/2123567310023848173
@SurendarR151771 let Karpathy's autoresearch run on a Mac overnight with nobody checking, and by morning it had run 116 experiments and kept 23. The reason it was safe to sleep is the design: the agent edits the training file, but the score comes from a file it can never touch. That one separation is what turns an overnight run from a gamble into a result.
πŸ’‘#7
@AiIvyagents
https://x.com/AiIvyagents/status/2123802339601490380
@AiIvyagents highlighted ARISE, where enterprise agents only accept a new rule when held-out accuracy improves, a gate that gained 10.9 points on SOP-Bench. The framing is the useful part: a self-improving agent needs a gate, not just a memory, because improvement without a standard is just drift. It is the same idea as the eval-script walls, applied to what the agent is allowed to learn rather than what it is allowed to change.
πŸ’‘#8
@TellersTech
https://x.com/TellersTech/status/2123837388627624346
@TellersTech shared a podcast clip on taking dbt 2 conformance from 45% to 93% with an AI agent loop. A conformance suite is close to the ideal autoresearch target: a fixed list of checks, a pass rate that moves in one direction, and no ambiguity about whether a change helped. Expect more migrations to be run this way, with the test suite as the metric and the agent grinding through failures.
πŸ’‘#9
@WrongNebula
https://x.com/WrongNebula/status/2123788310980288990
@WrongNebula posted a co-evolution loop for 3D modeling: generate an image, block out low-poly houses, project textures with Potemkin3D, detect depth and normals, feed that back into a custom harness, and repeat until the detail is good enough, then do the same for interiors. The author calls it Karpathy's autoresearch for modeling, and an earlier post showed the same loop on a single house. It is a good example of the recipe leaving code entirely, with depth and normal maps standing in for a validation loss.
πŸ’‘#10
@flaviusburca
https://x.com/flaviusburca/status/2123825516000272742
@flaviusburca did the arithmetic on alphaXiv's new OpenResearch index (end-to-end autoresearch, 8 models) right away: Opus 5.5 is 6.9 points better than GPT-6.1 Sol at 4.3x the price per task, and four runs of Sol cost $15.32, less than one run of Opus. For autoresearch loops, the author wants the index to report best-of-four at the same price point. That is the right question for a loop, where you get to try many times and only the best result counts.
πŸ’‘#11
@Secretof_forest
https://x.com/Secretof_forest/status/2123637245781619143
@Secretof_forest restated the alphaXiv index as cost per full 87-attempt run: Opus 5.5 at 0.618 for $1,430, GPT-6.1 Sol at 0.549 for $330, DeepSeek V4.1 Flash at 0.274 for $63. Sol gets 89% of Opus for 23% of the price. Framed that way, the index prices autoresearch rather than ranking it, and the choice of model becomes a budget decision.
πŸ’‘#12
@soubhikdeb
https://x.com/soubhikdeb/status/2123572450722267358
@soubhikdeb made two arguments about AI and science. First, research has never kept a ledger of negative results, so dead ends get re-explored by different people, and token costs now give labs a real incentive to record what did not work so their loops stop paying for it twice. Second, autoresearch forces verifiers to drop the hand-wavy parts that humans used to leave on intuition, because an AI reward-hacks far more efficiently than a person. Reward hacking, in this view, is a forcing function for rigor.
πŸ’‘#13
@kuan_yiii
https://x.com/kuan_yiii/status/2123825450128453740
@kuan_yiii closed a thread on an autoresearch project with a blunt remark: frontier models are still fragile at long-horizon autoresearch. The author had to hand-design the encoder and the gaze model just so any meaningful inductive bias survived into the decision transformer. The agent could iterate, but the structure that made the problem learnable still had to come from a person.
πŸ’‘#14
@xingchchoj37907
https://x.com/xingchchoj37907/status/2123735882088009857
@xingchchoj37907 is building a new autoresearch benchmark on one observation: LLM ability varies sharply across scientific fields, strong in math and weak in computational biology, even though neither needs a wet lab. A follow-up post adds that autoresearch is not the same as answering one scientific question, because it needs a better agent and more context across the whole research loop. If the gap is real, a single autoresearch score hides most of what a lab needs to know.
πŸ’‘#15
@boringdev_views
https://x.com/boringdev_views/status/2123479850728231103
@boringdev_views described what makes a self-improving agent useful: an evidence threshold. No proposal until a pattern repeats across trajectories, then the failures get replayed, and a human reviews the change before it ships. It is three gates in one sentence, and it is the opposite of the agents that rewrite their own skills after a single bad session.
πŸ’‘#16
@jatingargiitk
https://x.com/jatingargiitk/status/2123654765729161460
@jatingargiitk noticed that two READMEs are built around the same uncomfortable fact. Plannotator's Claude Code plan hook has a 345,600-second timeout, which is four days, and Agent Substrate snapshots idle agents out of memory because they spend most of their lives waiting on models, tools or humans. The slowest step in an agent loop is the person reviewing it, so the author now judges a setup on whether it can park, keep its state, and pick up a day later.
πŸ’‘#17
@zeromarkapp
https://x.com/zeromarkapp/status/2123491895762465230
@zeromarkapp turned Anthropic's claim that one agent loop with skills often beats subagent setups into a test anyone can run. Write 10 golden examples, merge one subagent into the main loop, and split it back if scores drop. The key is to check which tasks flipped rather than the total, because Anthropic's own test merge cut latency and broke a flow that used to pass.
πŸ’‘#18
@0x_gkn
https://x.com/0x_gkn/status/2123748821784248512
@0x_gkn has been running Antigravity with Gemini 4 and rates the output on par with Opus 5.5 for frontend, EDA and even infra. The recurring problem is in the loop, not the model: it often wraps up mid-task saying it will report back once tests or scans finish, and when those jobs complete, the session never follows up on its own. Async work is still where agent loops quietly drop the ball.
πŸ’‘#19
@Chakresh2005
https://x.com/Chakresh2005/status/2123832838374211660
@Chakresh2005 pointed out that slow deliberate reasoning looks great until you put it in an agent loop with tool use, where latency compounds every turn. The fix in their setup is a fast speculative model for tool routing, with the slow deliberate pass saved for the final evaluation and verification step. That split cut wall time by 4x.
πŸ’‘#20
@BenjieMalinao
https://x.com/BenjieMalinao/status/2123854787565477894
@BenjieMalinao flagged an OpenAI change that hits long-running loops: the agent's reasoning is now dropped if you edit earlier messages, which includes trimming old tool output, rewriting the system prompt, or swapping tools mid-run. On GPT-6.2 that means thinking twice and paying twice. The author's advice is to keep agent histories append-only, which turns a common context-management trick into a cost leak.
πŸ’‘#21
@leomaaaaaa
https://x.com/leomaaaaaa/status/2123624906449645694
@leomaaaaaa explained a $1.50 price gap between two runs as mostly a time gap: 41 minutes against 21 means nearly twice the agentic loop, and every extra minute burns more tokens. So the cheapest model is usually the fastest one that still finishes the job. Price per token matters less than minutes per task.
πŸ’‘#22
@Suryanshti777
https://x.com/Suryanshti777/status/2123857439787504045
@Suryanshti777 argued that a model picker inside an AI app builder is the new best-viewed-in-Internet-Explorer badge. Four frontier models shipped in nine days and Fable 5.2 alone came with four breaking API changes, so if your product owns the agent loop, your team eats every one of them. If the model calls your product instead, the upgrade lands in the user's chat and you ship nothing, and you stop losing margin every time a user picks a model that costs twice as much per token.
πŸ’‘#23
@yabhishekhd
https://x.com/yabhishekhd/status/2123674201420976357
@yabhishekhd summarized Perplexity's research on self-improving agent skills: the agent saves tasks you repeat as skills and reuses them next time. In Perplexity's own eight-week test, 72.9% of users got at least one skill made automatically, tool errors dropped 11% and unhappy chats dropped 7.6%. The author is careful to note these are the company's own numbers, but it is one of the few self-improving agent claims this week that came with any numbers at all.
πŸ’‘#24
@cleavey1985
https://x.com/cleavey1985/status/2123565788804333898
@cleavey1985 shared two blog posts from the first weeks at Modal. One turns a Sandbox into a Linux desktop, a browser or an Android phone and lets a model drive it with a short agent loop, with modal as the only dependency. The other walks through design choices for agents on Sandboxes: where the loop and the tools run, credentials, sidecars, networking and snapshots. Both come with code.
πŸ’‘#25
@DYoung_AI
https://x.com/DYoung_AI/status/2123655834148176074
@DYoung_AI wrote up a GitHub CLI change that matters for agent loops: from version 2.105.0, gh event listen waits for pull request changes instead of looping on gh pr view, and by default it exits after the first event, which is what a script or agent loop wants. It can wait for submitted reviews, state and head changes, merge conflicts and completed check suites, and requires Event Streams preview access. The author is upfront about not having joined the waitlist or tried it on a live PR yet.
πŸ’‘#26
@render
https://x.com/render/status/2123808855612342311
@render announced that Cursor Cloud Agents can now run on Render Sandboxes. Cursor handles the agent loop and inference, and Render gives each agent run its own isolated sandbox to write, run and test code. It is another sign that the loop and the place it executes are becoming separate products.
πŸ’‘#27
@ziv_ravid
https://x.com/ziv_ravid/status/2123896545401856114
@ziv_ravid relayed an investor's view on the AI bubble question: the growth is real, but early-stage neolabs look like a bubble. Close to a dozen auto-research companies have each raised more than $100M and none has revenue, and the expectation is one or two winners. Put next to the alphaXiv numbers, the open question is who pays $16 a task for research a loop produces.
πŸ’‘#28
@mike_pavlukhin
https://x.com/mike_pavlukhin/status/2123849848336306363
@mike_pavlukhin found that pairing Gemini 4 Carbon with Astra 6.2 works well for autoresearch. Astra on its own is stubborn and narrow, and Gemini helps it explore more directions. It is a small observation, but it matches the broader pattern of using one model to propose and another to push.
πŸ’‘#29
@wenhaocha1
https://x.com/wenhaocha1/status/2123814150422049264
@wenhaocha1 reported a better recipe after another small autoresearch run, faster and with higher PSNR, with the chart attached. Image-quality metrics like PSNR are exactly the kind of cheap, objective number the recipe needs, and small runs like this one are where most of the real use is happening.
πŸ’‘#30
@KomalVardhanL
https://x.com/KomalVardhanL/status/2123710576731869368
@KomalVardhanL is writing a running series on AutoResearch, and part 213 is a mistake worth copying: only use it on objective metrics like loss, accuracy, latency or pass rate. No automatic evaluation, no auto-research, and style and UX still need a human. The author learned it by burning a night on style first.
πŸ“‘ Eco Products Radar
Eco Products Radar

Karpathy's autoresearch: still the recipe people fork, this window applied to 3D modeling, overnight Mac runs, dbt migrations, virtual cell models and image pipelines.
Opus 5.5: top of the alphaXiv OpenResearch index at 61.8% and the most expensive per task, the yardstick every cost comparison was measured against.
GPT-6.1 Sol and GPT-6.2 Astra/Sol: the cheaper option on the index, the models in the rewound math benchmark, and the ones affected by OpenAI dropping reasoning on edited histories.
Gemini 4: driving Antigravity on par with Opus for frontend work, and pairing with Astra to widen exploration.
Claude Code: the host for plan-review hooks that wait four days, and one half of the Claude Code versus Codex architecture debates.
Codex: in the same harness comparisons, and the tool that was draining usage before the MITM forgetting proxy.
Hermes Agent: Nous Research's self-improving agent, pushed hard by promo posts this window and left out of the cases.
← Previous
Super User Daily: 2026-11-21
Next β†’
Ideas Radar: 2026-11-21
← Back to all articles

Comments

Loading...
>_