Loop Daily: April 25, 2026
The autoresearch and agentic-loop channels were quiet today — most of the chatter under those exact keywords was crypto noise or stray agent-commerce posts. The one piece worth reading in full is a new paper that lands squarely in the autoresearch-on-fine-tuning lane. An autonomous agent fine-tunes small language models end-to-end, and the numbers in cold-start mode push the upper bound of what this kind of loop can do without human hand-holding.
#1
@ash_csx
https://x.com/ash_csx/status/2047353838240408000
Published a paper on Pioneer, an autonomous fine-tuning agent that handles the entire lifecycle — task description in, fine-tuned model out — across eight benchmarks. The cold-start numbers are the headline. A Llama 3.2 3B base that could not even follow multiple-choice format on ARC-Challenge scored 5.3 percent, and Pioneer walked it to 72.6 percent over 11 iterations, with DeepSeek-R1 chain-of-thought traces as the decisive breakthrough. Qwen3 8B on HumanEval went to 92.7 percent pass-at-one in just four iterations, and they found that adding GPT-4.1-generated solutions hurt performance because external model outputs dilute the training signal. SMS spam classification with GLiNER2 went from 0.159 F1 to 0.997, with the last push from 0.98 requiring only 55 targeted examples. End-to-end runs completed in 8 to 12 hours at 12 to 55 dollars per run. They also shipped AdaptFT-Bench, a production-mode benchmark that mixes fixable noise with poisonous noise like false premises and label flips. On TriviaQA the agent beat naive retraining by 43 percentage points by the final stage. On GSM8K the Pioneer agent improved from 75.9 to 81.2 percent as noise accumulated, while naive retraining degraded from 71.6 to 64.7 percent. That last comparison is the loop story in one line — the agent gets better precisely where naive approaches get worse, because the agent is running diagnostic loops on its own failures and the naive pipeline is just shoveling contaminated data back in.
https://x.com/ash_csx/status/2047353838240408000
Published a paper on Pioneer, an autonomous fine-tuning agent that handles the entire lifecycle — task description in, fine-tuned model out — across eight benchmarks. The cold-start numbers are the headline. A Llama 3.2 3B base that could not even follow multiple-choice format on ARC-Challenge scored 5.3 percent, and Pioneer walked it to 72.6 percent over 11 iterations, with DeepSeek-R1 chain-of-thought traces as the decisive breakthrough. Qwen3 8B on HumanEval went to 92.7 percent pass-at-one in just four iterations, and they found that adding GPT-4.1-generated solutions hurt performance because external model outputs dilute the training signal. SMS spam classification with GLiNER2 went from 0.159 F1 to 0.997, with the last push from 0.98 requiring only 55 targeted examples. End-to-end runs completed in 8 to 12 hours at 12 to 55 dollars per run. They also shipped AdaptFT-Bench, a production-mode benchmark that mixes fixable noise with poisonous noise like false premises and label flips. On TriviaQA the agent beat naive retraining by 43 percentage points by the final stage. On GSM8K the Pioneer agent improved from 75.9 to 81.2 percent as noise accumulated, while naive retraining degraded from 71.6 to 64.7 percent. That last comparison is the loop story in one line — the agent gets better precisely where naive approaches get worse, because the agent is running diagnostic loops on its own failures and the naive pipeline is just shoveling contaminated data back in.
📡 Eco Products Radar
Eco Products Radar
Pioneer | Autonomous fine-tuning agent, cold-start to production
AdaptFT-Bench | Production-mode fine-tuning benchmark with poisonous noise
DeepSeek-R1 | Teacher-model source for CoT supervision inside Pioneer
Pioneer | Autonomous fine-tuning agent, cold-start to production
AdaptFT-Bench | Production-mode fine-tuning benchmark with poisonous noise
DeepSeek-R1 | Teacher-model source for CoT supervision inside Pioneer
Comments