Loop Daily: 2026-09-24
Two things happened in the same window and they point in opposite directions, which makes this the most honest week autoresearch has had. On the supply side it got institutional: OpenRSI shipped an open benchmark that turns real open-source projects into autoresearch environments, with 60-hour agent trajectories on 1,000-GPU clusters and over 100,000 H100-hours burned just to build the preview, and it publishes the traces rather than the scores. NVIDIA, NTU and MIT shipped SoL-Pi, where an auto-research loop proposed 152 harness changes and four survived, cutting token traffic by half at the harness layer with the same model underneath. Netflix is now hiring for this by name. On the demand side, two independent teams ran autoresearch against real financial data in the same window and both came back with the same verdict: the model has poor research taste. It proposes three nearly identical learning rates, barely explores the parameter space, and cannot judge a training curve. One of them beat the agent by hand, faster. What sits between those two results is the week's real theme, and several people named it independently: the loop only compounds when it has a deterministic gate on the other end. Strict external validation turned 103 noisy experiments into reproducible gains; without it, 95 percent of breakthroughs were overfits to noisy scrapers. Meanwhile the infrastructure conversation finally got the right vocabulary β the loop is not the sandbox, and a single-threaded loop in a code sandbox is to durability what a Java app on EC2 was to cloud in 2011.
#1
@OpenRSI
https://x.com/OpenRSI/status/2102831770458890626
Released OpenRSI-Index v0.1, an open benchmark for whether AI can recursively improve itself at production scale rather than in toy setups. Fully open-source projects get turned into autoresearch environments, with agent trajectories running 60+ hours on clusters of 1,000 GPUs. Building v0.1 alone consumed over 100,000 H100-hours. Everything is open including the traces: tasks, harnesses, verifiers and full agent trajectories, which is the part that matters if you have watched RSI claims go unverifiable all year.
https://x.com/OpenRSI/status/2102831770458890626
Released OpenRSI-Index v0.1, an open benchmark for whether AI can recursively improve itself at production scale rather than in toy setups. Fully open-source projects get turned into autoresearch environments, with agent trajectories running 60+ hours on clusters of 1,000 GPUs. Building v0.1 alone consumed over 100,000 H100-hours. Everything is open including the traces: tasks, harnesses, verifiers and full agent trajectories, which is the part that matters if you have watched RSI claims go unverifiable all year.
#2
@yuetai12575
https://x.com/yuetai12575/status/2102884893562941492
The project initiator explains the reasoning behind OpenRSI-Index, and the honest part is the admission that everyone is excited about RSI while the barriers to participation keep everyone outside the door. Their answer is RSI-Anything, a human-AI pipeline that turns a real research question into a runnable autoresearch environment in about an hour of conversation. Tasks come from real projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab. His framing of why RSI matters is the cleanest in the thread: human insights are limited by bandwidth, agents can scale idea generation and sweep a far larger region of method-space.
https://x.com/yuetai12575/status/2102884893562941492
The project initiator explains the reasoning behind OpenRSI-Index, and the honest part is the admission that everyone is excited about RSI while the barriers to participation keep everyone outside the door. Their answer is RSI-Anything, a human-AI pipeline that turns a real research question into a runnable autoresearch environment in about an hour of conversation. Tasks come from real projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab. His framing of why RSI matters is the cleanest in the thread: human insights are limited by bandwidth, agents can scale idea generation and sweep a far larger region of method-space.
#3
@ayyazdev
https://x.com/ayyazdev/status/2102883543479173512
The clearest writeup of NVIDIA, NTU and MIT's SoL-Pi, an MIT-licensed extension that sits at the harness layer with the same model underneath. On EdgeBench's 51 tasks it cut recorded token traffic 44.7 to 49 percent versus stock Pi and dropped API cost about a third, holding roughly 94 percent of Pi's score on GPT-5.6 Sol and Opus 5. The mechanism is the story: an auto-research loop proposed 152 harness changes and only four survived β Action Fusion, ObservationPack, Online Context Compact, and an Evidence-Preserving Reducer that ships fat build and test logs through a cheaper model with a verifier. His closing line lands: we have all been shopping models hard, and a chunk of the bill is just how the agent talks to tools.
https://x.com/ayyazdev/status/2102883543479173512
The clearest writeup of NVIDIA, NTU and MIT's SoL-Pi, an MIT-licensed extension that sits at the harness layer with the same model underneath. On EdgeBench's 51 tasks it cut recorded token traffic 44.7 to 49 percent versus stock Pi and dropped API cost about a third, holding roughly 94 percent of Pi's score on GPT-5.6 Sol and Opus 5. The mechanism is the story: an auto-research loop proposed 152 harness changes and only four survived β Action Fusion, ObservationPack, Online Context Compact, and an Evidence-Preserving Reducer that ships fat build and test logs through a cheaper model with a verifier. His closing line lands: we have all been shopping models hard, and a chunk of the bill is just how the agent talks to tools.
#4
@ayyazdev
https://x.com/ayyazdev/status/2102796195521335395
The follow-up detail from SoL-Pi's tables that most coverage skipped: on Sol, token spend went from 2.15B tokens and $1,339 down to 1.10B and $894. But the odd bit is that ObservationPack alone scored 47.2 against plain Pi's 44.8, meaning the full cheap stack is not automatically the best-scoring configuration. That is a real finding about auto-research output β the loop optimizes for the objective you gave it, and stacking all four survivors is a cost decision, not a quality one.
https://x.com/ayyazdev/status/2102796195521335395
The follow-up detail from SoL-Pi's tables that most coverage skipped: on Sol, token spend went from 2.15B tokens and $1,339 down to 1.10B and $894. But the odd bit is that ObservationPack alone scored 47.2 against plain Pi's 44.8, meaning the full cheap stack is not automatically the best-scoring configuration. That is a real finding about auto-research output β the loop optimizes for the objective you gave it, and stacking all four survivors is a cost decision, not a quality one.
#5
@xpasky
https://x.com/xpasky/status/2102880072474538022
One image and one sentence that does more work than most benchmark threads: Astra a few days into an autoresearch loop versus Opus 5.5 after about six hours of work. The comparison people keep making is model against model at a fixed moment. The comparison that actually matters for autoresearch is what a loop looks like after it has been allowed to run, and that is a different axis entirely.
https://x.com/xpasky/status/2102880072474538022
One image and one sentence that does more work than most benchmark threads: Astra a few days into an autoresearch loop versus Opus 5.5 after about six hours of work. The comparison people keep making is model against model at a fixed moment. The comparison that actually matters for autoresearch is what a loop looks like after it has been allowed to run, and that is a different axis entirely.
#6
@jaredpalmer
https://x.com/jaredpalmer/status/2102818062512554153
Running autoresearch on Kev overnight using Devin's cloud agent with Modal Outposts for full GPU access, plus Devin's macOS VMs for MLX work. He calls the workflow super powerful and then does the useful thing and names the gap: the automations need to score PRs and issues before blindly taking action and trying to reproduce, and the same applies to AutoReview. His closing observation generalizes beyond his repo β open source has a different security and trust model for agents than internal work does.
https://x.com/jaredpalmer/status/2102818062512554153
Running autoresearch on Kev overnight using Devin's cloud agent with Modal Outposts for full GPU access, plus Devin's macOS VMs for MLX work. He calls the workflow super powerful and then does the useful thing and names the gap: the automations need to score PRs and issues before blindly taking action and trying to reproduce, and the same applies to AutoReview. His closing observation generalizes beyond his repo β open source has a different security and trust model for agents than internal work does.
#7
@EMostaque
https://x.com/EMostaque/status/2102810115703484479
Points at the Bespoke Labs writeup on the Autoresearch exam and pulls out the finding that should reset expectations: it shows minimal difference in performance across standard harnesses. If that holds, the harness-obsession of the past two months has a ceiling, and his read is that the reliable frontier is within reach for everyone rather than being a function of who has the best scaffolding.
https://x.com/EMostaque/status/2102810115703484479
Points at the Bespoke Labs writeup on the Autoresearch exam and pulls out the finding that should reset expectations: it shows minimal difference in performance across standard harnesses. If that holds, the harness-obsession of the past two months has a ceiling, and his read is that the reliable frontier is within reach for everyone rather than being a function of who has the best scaffolding.
#8
@PhungVanDuy1
https://x.com/PhungVanDuy1/status/2102804744393920885
The cost problem with running /goal, agent swarms, or both at once is that frontier models at scale get expensive fast. Their experiments suggest DeepSeek Flash 4.1 delivers comparable quality at a fraction of the cost, and he says the result genuinely surprised him. They are continuing to push Auto Research plus agent swarm scaling inside Meta Zenith with the explicit constraint of keeping inference costs practical, which is the constraint most swarm demos quietly ignore.
https://x.com/PhungVanDuy1/status/2102804744393920885
The cost problem with running /goal, agent swarms, or both at once is that frontier models at scale get expensive fast. Their experiments suggest DeepSeek Flash 4.1 delivers comparable quality at a fraction of the cost, and he says the result genuinely surprised him. They are continuing to push Auto Research plus agent swarm scaling inside Meta Zenith with the explicit constraint of keeping inference costs practical, which is the constraint most swarm demos quietly ignore.
#9
@edotenv
https://x.com/edotenv/status/2102075985973690493
The most valuable negative result of the week, and it is specific. They let GPT autoresearch deep learning alphas on crypto perp data and found three concrete failure modes: it does not explore parameters well, making only minimal changes across iterations; it has poor research taste, proposing experiments like three almost identical learning rates; and it struggles with intelligent evals, particularly judging training curves where there is no binary criterion. His conclusion is that there is still a long way to go before LLMs are useful for genuinely hard research tasks.
https://x.com/edotenv/status/2102075985973690493
The most valuable negative result of the week, and it is specific. They let GPT autoresearch deep learning alphas on crypto perp data and found three concrete failure modes: it does not explore parameters well, making only minimal changes across iterations; it has poor research taste, proposing experiments like three almost identical learning rates; and it struggles with intelligent evals, particularly judging training curves where there is no binary criterion. His conclusion is that there is still a long way to go before LLMs are useful for genuinely hard research tasks.
#10
@RRicefan
https://x.com/RRicefan/status/2102077726563971190
The same negative result from a second team, with the sharpest possible framing. Autoresearch with GPT on real historical data and a high-ish frequency target shows GPT is a bad researcher with poor research taste. The kicker is the comparison: his own manually fitted models outperform GPT's auto-researched ones by far, and took him less time. Two independent teams landing on research taste as the bottleneck in the same window is the signal here, not either result alone.
https://x.com/RRicefan/status/2102077726563971190
The same negative result from a second team, with the sharpest possible framing. Autoresearch with GPT on real historical data and a high-ish frequency target shows GPT is a bad researcher with poor research taste. The kicker is the comparison: his own manually fitted models outperform GPT's auto-researched ones by far, and took him less time. Two independent teams landing on research taste as the bottleneck in the same window is the signal here, not either result alone.
#11
@Kizuno18
https://x.com/Kizuno18/status/2101834245471707397
The counterweight to those two, and it names the mechanism rather than just asserting optimism. Running an unmonitored pipeline on Brazil demographic data, when models hypothesize and test data treatments, 95 percent of breakthroughs turn out to be subtle overfits to noisy scrapers. His conclusion is that the real compounding only starts when autoresearch loops get deterministic evaluation gates: strict external validation turned 103 noisy experiments into real, reproducible gains.
https://x.com/Kizuno18/status/2101834245471707397
The counterweight to those two, and it names the mechanism rather than just asserting optimism. Running an unmonitored pipeline on Brazil demographic data, when models hypothesize and test data treatments, 95 percent of breakthroughs turn out to be subtle overfits to noisy scrapers. His conclusion is that the real compounding only starts when autoresearch loops get deterministic evaluation gates: strict external validation turned 103 noisy experiments into real, reproducible gains.
#12
@ChrisJMcCormick
https://x.com/ChrisJMcCormick/status/2102102027279204747
A working example of the useful failure mode of autoresearch: he lifted specific ideas out of a published auto-research run rather than the whole thing. What he borrowed is concrete β batch size of 192K, gates on x0 lambdas, two huge bigram hash tables of 2M and 1M rows optimized sparse with no first moment and only per-row magnitude. What he deliberately did not borrow is the FP8 and custom triton ops, and he notes his codebase is still clean rather than Clauded-up. Baseline 4 cut time by 40 percent, and the two changes he flags as fun are dropping .grad buffers from the hash tables to inline the optimizer in their backward math, and Fable finding the right spelling of ReLU squared backward so Inductor would fuse it correctly.
https://x.com/ChrisJMcCormick/status/2102102027279204747
A working example of the useful failure mode of autoresearch: he lifted specific ideas out of a published auto-research run rather than the whole thing. What he borrowed is concrete β batch size of 192K, gates on x0 lambdas, two huge bigram hash tables of 2M and 1M rows optimized sparse with no first moment and only per-row magnitude. What he deliberately did not borrow is the FP8 and custom triton ops, and he notes his codebase is still clean rather than Clauded-up. Baseline 4 cut time by 40 percent, and the two changes he flags as fun are dropping .grad buffers from the hash tables to inline the optimizer in their backward math, and Fable finding the right spelling of ReLU squared backward so Inductor would fuse it correctly.
#13
@gregpr07
https://x.com/gregpr07/status/2102157421909258560
One night of autoresearch that did not generalize, reported honestly, which is more useful than the ones that did. It tried a lot of crazy stuff optimizing vLLM: caching optimized specifically for a custom harness, and a diffusion transformer implemented on a 2048 block that made inference 17x faster. Both are real results. Neither transferred. That gap between a spectacular local win and something that holds outside the setup it was found in is the actual open problem in this field.
https://x.com/gregpr07/status/2102157421909258560
One night of autoresearch that did not generalize, reported honestly, which is more useful than the ones that did. It tried a lot of crazy stuff optimizing vLLM: caching optimized specifically for a custom harness, and a diffusion transformer implemented on a 2048 block that made inference 17x faster. Both are real results. Neither transferred. That gap between a spectacular local win and something that holds outside the setup it was found in is the actual open problem in this field.
#14
@AtaeiMe
https://x.com/AtaeiMe/status/2102033799424954723
The most substantive argument of the week for making autoresearch traces a first-class artifact. His premise is that ICLR is around 60k submission IDs this cycle and producing a paper keeps getting cheaper while checking one still takes time, so the old publishing system is going away and we should work out what we want after it rather than trying to restore it. His proposal: universities should run auto-research openly with the traces available alongside the paper, so you can see what was tried and why someone gave up on it, and if a person spotted a bad assumption and redirected the work, that should be recorded. Publishing already worked like decentralized recursive self-improvement; papers compressed that process and left a lot out. With automated research we can keep the attempts and corrections too.
https://x.com/AtaeiMe/status/2102033799424954723
The most substantive argument of the week for making autoresearch traces a first-class artifact. His premise is that ICLR is around 60k submission IDs this cycle and producing a paper keeps getting cheaper while checking one still takes time, so the old publishing system is going away and we should work out what we want after it rather than trying to restore it. His proposal: universities should run auto-research openly with the traces available alongside the paper, so you can see what was tried and why someone gave up on it, and if a person spotted a bad assumption and redirected the work, that should be recorded. Publishing already worked like decentralized recursive self-improvement; papers compressed that process and left a lot out. With automated research we can keep the attempts and corrections too.
#15
@cHHillee
https://x.com/cHHillee/status/2102161884740977120
Names the three properties an autoresearch loop actually needs from its infrastructure, which is more useful than another opinion about whether RSI is real. It should be unbounded by your practical compute constraints, autoscale up and down instantly and trivially, and have very quick iteration loops. He thinks Tinker is a good fit for exactly those reasons. Notice that none of the three is about model quality.
https://x.com/cHHillee/status/2102161884740977120
Names the three properties an autoresearch loop actually needs from its infrastructure, which is more useful than another opinion about whether RSI is real. It should be unbounded by your practical compute constraints, autoscale up and down instantly and trivially, and have very quick iteration loops. He thinks Tinker is a good fit for exactly those reasons. Notice that none of the three is about model quality.
#16
@DimitrisPapail
https://x.com/DimitrisPapail/status/2102087601775886547
A precise result about whether multi-agent communication actually beats just running one agent longer. He frames the question as the horizontal gap between the team curve and the best-of-N curve between T and NT, and reports that communication still wins: in autoresearch for model compression, the team reaches lower KBs early, and a single agent given 10 to 100x more time is not closing the gap. A team of four is the comparison line.
https://x.com/DimitrisPapail/status/2102087601775886547
A precise result about whether multi-agent communication actually beats just running one agent longer. He frames the question as the horizontal gap between the team curve and the best-of-N curve between T and NT, and reports that communication still wins: in autoresearch for model compression, the team reaches lower KBs early, and a single agent given 10 to 100x more time is not closing the gap. A team of four is the comparison line.
#17
@sytelus
https://x.com/sytelus/status/2102055500565082599
A concrete prediction with a stated mechanism, which is rare in this genre. He argues the biggest impact of mathematics will not be the millennium problems but proofs in general software at scale, and that all mainstream software will be verified by 2030 β you likely will not touch an unverified library. The reason he names formalization of specs and doing this at 1B LoC scale as ripe targets for auto-research and self-improvement loops is that both are bulk, mechanical and verifiable, which is exactly the shape autoresearch is good at.
https://x.com/sytelus/status/2102055500565082599
A concrete prediction with a stated mechanism, which is rare in this genre. He argues the biggest impact of mathematics will not be the millennium problems but proofs in general software at scale, and that all mainstream software will be verified by 2030 β you likely will not touch an unverified library. The reason he names formalization of specs and doing this at 1B LoC scale as ripe targets for auto-research and self-improvement loops is that both are bulk, mechanical and verifiable, which is exactly the shape autoresearch is good at.
#18
@iamMrDuncan
https://x.com/iamMrDuncan/status/2101920242557399105
A live autoresearch run reported with its baseline, which is how these should be posted. Starting a new run on a P100 for ELX3 Qwen 3.8 27b, and he publishes the baseline screenshot up front and calls it horrendous himself: 10.16 tokens per second on Pascal with an FP16 card running low quants. The target is what 24 hours of autoresearch using Qwen 3.8 Flash on 2x DGX Sparks can do to that number. Publishing the ugly starting number before the run is the part worth copying.
https://x.com/iamMrDuncan/status/2101920242557399105
A live autoresearch run reported with its baseline, which is how these should be posted. Starting a new run on a P100 for ELX3 Qwen 3.8 27b, and he publishes the baseline screenshot up front and calls it horrendous himself: 10.16 tokens per second on Pascal with an FP16 card running low quants. The target is what 24 hours of autoresearch using Qwen 3.8 Flash on 2x DGX Sparks can do to that number. Publishing the ugly starting number before the run is the part worth copying.
#19
@Tech_girl
https://x.com/Tech_girl/status/2102111778624692600
A head-to-head on Karpathy's autoresearch benchmark that is worth reading for the shape of the number rather than the winner. On SylphAI's A10 test, AdaL ran 336 experiments against Claude Code's 76, and also reached a better final BPB. Four times the experiment throughput on the same hardware is a harness-and-orchestration result, not a model result, and the raw results are public.
https://x.com/Tech_girl/status/2102111778624692600
A head-to-head on Karpathy's autoresearch benchmark that is worth reading for the shape of the number rather than the winner. On SylphAI's A10 test, AdaL ran 336 experiments against Claude Code's 76, and also reached a better final BPB. Four times the experiment throughput on the same hardware is a harness-and-orchestration result, not a model result, and the raw results are public.
#20
@MimansaJ
https://x.com/MimansaJ/status/2102845528447004804
A hiring post that doubles as a market signal: Netflix's member foundation models team is hiring a Winter '26 PhD research intern to build LLM-driven AutoResearch agents and harness for improving traditional recommendation models. Autoresearch showing up as a headcount line at a company whose core problem is recommendations, not frontier models, is a better indicator of the technique escaping the lab than any benchmark this week.
https://x.com/MimansaJ/status/2102845528447004804
A hiring post that doubles as a market signal: Netflix's member foundation models team is hiring a Winter '26 PhD research intern to build LLM-driven AutoResearch agents and harness for improving traditional recommendation models. Autoresearch showing up as a headcount line at a company whose core problem is recommendations, not frontier models, is a better indicator of the technique escaping the lab than any benchmark this week.
#21
@GiulioRebuffo
https://x.com/GiulioRebuffo/status/2102839434534166755
A working artifact rather than a claim: he built a home-grown standard library substitute for Bend, covering early math functions, core data structures and cryptography including blake3, keccak and sha, and says everything was formally verified to the best of his ability. The benchmarks are already striking β blake3 is sometimes faster than C, most data structure operations land under 2.5x C, and his own hash table implementation beats Base.Bend by 10 to 20x depending on load. Then the sign-off that makes it a Loop story: some benchmarks still suck, be patient, AutoRESEARCH is running.
https://x.com/GiulioRebuffo/status/2102839434534166755
A working artifact rather than a claim: he built a home-grown standard library substitute for Bend, covering early math functions, core data structures and cryptography including blake3, keccak and sha, and says everything was formally verified to the best of his ability. The benchmarks are already striking β blake3 is sometimes faster than C, most data structure operations land under 2.5x C, and his own hash table implementation beats Base.Bend by 10 to 20x depending on load. Then the sign-off that makes it a Loop story: some benchmarks still suck, be patient, AutoRESEARCH is running.
#22
@joshliusg
https://x.com/joshliusg/status/2102826838490075355
States the standard the field should be held to and then shows receipts for it: RSI claims only count when they are verifiable, and open benchmarks plus open code is the right bar. AutoTrust's ScienceGuru has put up two public RSI results β Autoresearch@Home number one, and NanoPath v2 number one validated β with recipes on GitHub. The reason this matters is that the gap between claimed and reproducible RSI has been the whole credibility problem, and validated is doing real work in that sentence.
https://x.com/joshliusg/status/2102826838490075355
States the standard the field should be held to and then shows receipts for it: RSI claims only count when they are verifiable, and open benchmarks plus open code is the right bar. AutoTrust's ScienceGuru has put up two public RSI results β Autoresearch@Home number one, and NanoPath v2 number one validated β with recipes on GitHub. The reason this matters is that the gap between claimed and reproducible RSI has been the whole credibility problem, and validated is doing real work in that sentence.
#23
@johnsaigle
https://x.com/johnsaigle/status/2102867057511682285
One sentence that is a complete product idea: the answer to agents writing no-op and tautology tests might be mutation testing run as an autoresearch loop. It is the right shape because mutation testing supplies exactly what an autoresearch loop needs and a test suite normally lacks β an automatic, objective signal for whether a test actually catches anything.
https://x.com/johnsaigle/status/2102867057511682285
One sentence that is a complete product idea: the answer to agents writing no-op and tautology tests might be mutation testing run as an autoresearch loop. It is the right shape because mutation testing supplies exactly what an autoresearch loop needs and a test suite normally lacks β an automatic, objective signal for whether a test actually catches anything.
#24
@JamesWard
https://x.com/JamesWard/status/2102468057494917237
A precise report on where an LLM is still load-bearing inside a working agentic loop after you strip out everything a decision model can handle. For his Jev-based loop there are now only two places he needs an LLM as a tool: summarization and parameter parsing. He used to use the LLM to create filter criteria for items returned from tool calls, the way a coding assistant defines grep and find calls, but now Jev just does the filtering directly. His observation is worth keeping: Jev cannot author a filter criterion, but for small enough data sets doing the semantic filter directly is probably more accurate than a generated criterion.
https://x.com/JamesWard/status/2102468057494917237
A precise report on where an LLM is still load-bearing inside a working agentic loop after you strip out everything a decision model can handle. For his Jev-based loop there are now only two places he needs an LLM as a tool: summarization and parameter parsing. He used to use the LLM to create filter criteria for items returned from tool calls, the way a coding assistant defines grep and find calls, but now Jev just does the filtering directly. His observation is worth keeping: Jev cannot author a filter criterion, but for small enough data sets doing the semantic filter directly is probably more accurate than a generated criterion.
#25
@salman_paracha
https://x.com/salman_paracha/status/2102818700231655463
Introduces a useful piece of vocabulary: Harness Runtime manages the agentic loop, or more precisely it manages the harness running the loop. He calls it a meta-harness that maintains agent sessions and scales more work, with the agent still able to use local shell commands and file access. Naming the layer above the harness matters because most of this week's arguments about which harness is best are actually arguments about the layer nobody had a word for.
https://x.com/salman_paracha/status/2102818700231655463
Introduces a useful piece of vocabulary: Harness Runtime manages the agentic loop, or more precisely it manages the harness running the loop. He calls it a meta-harness that maintains agent sessions and scales more work, with the agent still able to use local shell commands and file access. Naming the layer above the harness matters because most of this week's arguments about which harness is best are actually arguments about the layer nobody had a word for.
#26
@salman_paracha
https://x.com/salman_paracha/status/2102620784862917009
Follows up with what a sandbox actually means in this architecture, and it is a genuinely clarifying answer. The Harness Runtime ships with some local tools like chromium for convenience, but code execution isolation is scaled through a single tool called do.action_code. The agent can run simple shell commands locally for filesystem access or to call host utilities, but if the model-directed work needs arbitrary code execution such as a Monte Carlo simulation, do.action_code spins up an additional microVM with the right language dependencies and returns output via artifacts and stdout for the calling agent to read and proceed. Isolation as a tool call rather than as a property of the whole environment is the design decision worth noting.
https://x.com/salman_paracha/status/2102620784862917009
Follows up with what a sandbox actually means in this architecture, and it is a genuinely clarifying answer. The Harness Runtime ships with some local tools like chromium for convenience, but code execution isolation is scaled through a single tool called do.action_code. The agent can run simple shell commands locally for filesystem access or to call host utilities, but if the model-directed work needs arbitrary code execution such as a Monte Carlo simulation, do.action_code spins up an additional microVM with the right language dependencies and returns output via artifacts and stdout for the calling agent to read and proceed. Isolation as a tool call rather than as a property of the whole environment is the design decision worth noting.
#27
@mfateev
https://x.com/mfateev/status/2102552442714063053
A short endorsement from Temporal that carries more weight than its length: running the agentic loop outside the sandbox as durable execution is a very common pattern they see. The sandbox is where code runs; the loop is a workflow that has to survive crashes, restarts and multi-day horizons. Conflating those two is the mistake, and a durable-execution vendor saying they see this pattern constantly is the strongest available evidence that it is the right decomposition.
https://x.com/mfateev/status/2102552442714063053
A short endorsement from Temporal that carries more weight than its length: running the agentic loop outside the sandbox as durable execution is a very common pattern they see. The sandbox is where code runs; the loop is a workflow that has to survive crashes, restarts and multi-day horizons. Conflating those two is the mistake, and a durable-execution vendor saying they see this pattern constantly is the strongest available evidence that it is the right decomposition.
#28
@ebarroca
https://x.com/ebarroca/status/2101902058697769062
The same point compressed into the week's best analogy: a single-threaded agentic loop in a code sandbox is to durability what a Java app on EC2 was to cloud in 2011. Sandboxes are a tool. They are not the runtime. Anyone who lived through that era knows exactly what the next five years of this look like.
https://x.com/ebarroca/status/2101902058697769062
The same point compressed into the week's best analogy: a single-threaded agentic loop in a code sandbox is to durability what a Java app on EC2 was to cloud in 2011. Sandboxes are a tool. They are not the runtime. Anyone who lived through that era knows exactly what the next five years of this look like.
#29
@codemarchant
https://x.com/codemarchant/status/2102736188411179417
The operational cost of an unbounded loop, stated with a number. He woke up to a $55 bill on a single Grok 4.7 API background request, because it got stuck in a 17-minute agentic loop doing web research on a basic request and he had not set max_turns to a reasonable number. Every architectural conversation about termination conditions this week has this tweet as its concrete counterpart.
https://x.com/codemarchant/status/2102736188411179417
The operational cost of an unbounded loop, stated with a number. He woke up to a $55 bill on a single Grok 4.7 API background request, because it got stuck in a 17-minute agentic loop doing web research on a basic request and he had not set max_turns to a reasonable number. Every architectural conversation about termination conditions this week has this tweet as its concrete counterpart.
#30
@OnFinality
https://x.com/OnFinality/status/2102413942824075551
Asks the question that separates people who have shipped an agentic loop from people who have read about one. The loop itself is easy; deciding when it should stop, retry, or hand back to a human is the part that eats the time. He aims it at a certification course that spends most of its hours on agentic architecture and orchestration, and asks whether it covers termination conditions at all. It is a fair test for any curriculum in this space.
https://x.com/OnFinality/status/2102413942824075551
Asks the question that separates people who have shipped an agentic loop from people who have read about one. The loop itself is easy; deciding when it should stop, retry, or hand back to a human is the part that eats the time. He aims it at a certification course that spends most of its hours on agentic architecture and orchestration, and asks whether it covers termination conditions at all. It is a fair test for any curriculum in this space.
#31
@HarishTeens
https://x.com/HarishTeens/status/2102312251072070005
One line that is a genuine contrarian take worth arguing with: if your service gets a ton of issues every day, building a self-healing agentic loop to auto-fix them is a bad idea. The implicit argument is that a high issue rate is information about your system, and a loop that silently absorbs it removes the pressure that would otherwise force you to fix the cause. Self-healing and root-cause analysis are in tension, and almost nobody in the swarm discourse is saying so.
https://x.com/HarishTeens/status/2102312251072070005
One line that is a genuine contrarian take worth arguing with: if your service gets a ton of issues every day, building a self-healing agentic loop to auto-fix them is a bad idea. The implicit argument is that a high issue rate is information about your system, and a loop that silently absorbs it removes the pressure that would otherwise force you to fix the cause. Self-healing and root-cause analysis are in tension, and almost nobody in the swarm discourse is saying so.
#32
@josh_garrett_kc
https://x.com/josh_garrett_kc/status/2102097716432060511
A specific behavioural observation from benchmarking, which is more useful than the headline. He gave Astra a reference image and a basic request for a Unity hand sanitizer station, deliberately allowing more agency than usual. The generation was good, but the thing he had not seen before in his benchmarking was that it showed actual accountability: it noticed a mistake it had made and self-addressed it in the agentic loop's next steps. Self-correction inside the loop without being told is the behaviour worth tracking here, not the asset.
https://x.com/josh_garrett_kc/status/2102097716432060511
A specific behavioural observation from benchmarking, which is more useful than the headline. He gave Astra a reference image and a basic request for a Unity hand sanitizer station, deliberately allowing more agency than usual. The generation was good, but the thing he had not seen before in his benchmarking was that it showed actual accountability: it noticed a mistake it had made and self-addressed it in the agentic loop's next steps. Self-correction inside the loop without being told is the behaviour worth tracking here, not the asset.
#33
@serrynaimo
https://x.com/serrynaimo/status/2102172448238235948
A quantified regression report on quantization, which is the kind of thing that almost never gets posted. He reverted from Splash: the performance is quite good, but the Splash quant gets stuck in agentic loop issues about 30 percent more often than MTPLX Optimized Speed. Raw throughput and loop-completion reliability are different axes, and this is one of the few numbers anyone has put on the gap.
https://x.com/serrynaimo/status/2102172448238235948
A quantified regression report on quantization, which is the kind of thing that almost never gets posted. He reverted from Splash: the performance is quite good, but the Splash quant gets stuck in agentic loop issues about 30 percent more often than MTPLX Optimized Speed. Raw throughput and loop-completion reliability are different axes, and this is one of the few numbers anyone has put on the gap.
#34
@serrynaimo
https://x.com/serrynaimo/status/2102629884061450309
A good example of what optimizing the loop rather than the model buys you on consumer hardware. He assembled his own llama.cpp build for an AMD card because it had to run weeks of knowledge-base work and report publishing. Between agentic loop optimization and good prefill and decode while maintaining enough quality, he got the runtime down from weeks to days.
https://x.com/serrynaimo/status/2102629884061450309
A good example of what optimizing the loop rather than the model buys you on consumer hardware. He assembled his own llama.cpp build for an AMD card because it had to run weeks of knowledge-base work and report publishing. Between agentic loop optimization and good prefill and decode while maintaining enough quality, he got the runtime down from weeks to days.
#35
@kurtbuhler
https://x.com/kurtbuhler/status/2102277120923697184
The most thought-through argument this week for where a decision model belongs inside an agentic loop, and he gives three positions rather than one. At the front to facilitate decision-making, in the middle to choose among options assuming those options are finite and the criteria are somewhat objective and measurable, and at the end as evaluation, again presuming the criteria are objective and measurable. The caveat is doing real work in that sentence: everything he proposes is gated on measurable criteria, which is exactly the constraint that separates this from the swarm hype.
https://x.com/kurtbuhler/status/2102277120923697184
The most thought-through argument this week for where a decision model belongs inside an agentic loop, and he gives three positions rather than one. At the front to facilitate decision-making, in the middle to choose among options assuming those options are finite and the criteria are somewhat objective and measurable, and at the end as evaluation, again presuming the criteria are objective and measurable. The caveat is doing real work in that sentence: everything he proposes is gated on measurable criteria, which is exactly the constraint that separates this from the swarm hype.
#36
@proxy_vector
https://x.com/proxy_vector/status/2102203959654195324
The single best question asked of anyone claiming an unsupervised agentic loop this week: what is the biggest failure mode you have hit running it unsupervised for a full workday. It is the question that separates a demo from an operating system, and the fact that it needed asking is itself the finding.
https://x.com/proxy_vector/status/2102203959654195324
The single best question asked of anyone claiming an unsupervised agentic loop this week: what is the biggest failure mode you have hit running it unsupervised for a full workday. It is the question that separates a demo from an operating system, and the fact that it needed asking is itself the finding.
π‘ Eco Products Radar
Eco Products Radar
SoL-Pi β NVIDIA/NTU/MIT harness-layer extension for the Pi coding agent, MIT licensed, produced by an auto-research loop over 152 candidate changes.
OpenRSI-Index β open benchmark turning open-source projects into autoresearch environments, plus the RSI-Anything pipeline that packages a research question into a runnable task in about an hour.
Pi β the baseline coding agent everything in the efficiency thread is measured against.
Jev β the decision-model layer, now appearing inside agentic loops for routing, filtering and evaluation rather than generation.
Claude Code β the comparison harness in the autoresearch benchmark results, and the one AdaL ran 336 experiments against 76 on.
Devin β cloud agent used for overnight autoresearch runs with Modal Outposts for GPU access and macOS VMs for MLX work.
Temporal β durable execution, and the vendor confirming that running the agentic loop outside the sandbox is now a common pattern.
EdgeBench β the 51-task benchmark the harness efficiency numbers are all reported on.
Harness Runtime β the emerging name for the meta-harness layer that manages the harness managing the loop.
SoL-Pi β NVIDIA/NTU/MIT harness-layer extension for the Pi coding agent, MIT licensed, produced by an auto-research loop over 152 candidate changes.
OpenRSI-Index β open benchmark turning open-source projects into autoresearch environments, plus the RSI-Anything pipeline that packages a research question into a runnable task in about an hour.
Pi β the baseline coding agent everything in the efficiency thread is measured against.
Jev β the decision-model layer, now appearing inside agentic loops for routing, filtering and evaluation rather than generation.
Claude Code β the comparison harness in the autoresearch benchmark results, and the one AdaL ran 336 experiments against 76 on.
Devin β cloud agent used for overnight autoresearch runs with Modal Outposts for GPU access and macOS VMs for MLX work.
Temporal β durable execution, and the vendor confirming that running the agentic loop outside the sandbox is now a common pattern.
EdgeBench β the 51-task benchmark the harness efficiency numbers are all reported on.
Harness Runtime β the emerging name for the meta-harness layer that manages the harness managing the loop.
Comments