A Paper That Pinpoints Exactly Where Your Research Agent Screwed Up
Topping HuggingFace's daily papers today is a deceptively practical piece of work from NJU-LINK Lab at Nanjing University. The question it asks: when a deep-research agent gives you a wrong answer, knowing that it failed is useless. You need to know where it failed.
The approach is span-level error localization in agent trajectories. Instead of marking a whole multi-step run as pass or fail, they break the trajectory into segments and pinpoint the exact span where the error first crept in. Did the agent pull a bad source on step three, or did it read the right source correctly and then reason wrong four steps later? Those are completely different bugs and today most eval setups cannot tell them apart.
This matters more than it sounds, and the 44 upvotes suggest the community agrees. As agents run longer and longer trajectories, the standard way of debugging, re-reading the entire transcript by hand, just does not scale. Span-level attribution is basically a diff view for agent failures. It points your eye at the line that broke instead of making you read the whole novel.
If you build or evaluate research agents, this is the kind of unglamorous tooling work that quietly raises the ceiling on everyone. Paper at https://arxiv.org/abs/2606.02060
← Back to all articles
The approach is span-level error localization in agent trajectories. Instead of marking a whole multi-step run as pass or fail, they break the trajectory into segments and pinpoint the exact span where the error first crept in. Did the agent pull a bad source on step three, or did it read the right source correctly and then reason wrong four steps later? Those are completely different bugs and today most eval setups cannot tell them apart.
This matters more than it sounds, and the 44 upvotes suggest the community agrees. As agents run longer and longer trajectories, the standard way of debugging, re-reading the entire transcript by hand, just does not scale. Span-level attribution is basically a diff view for agent failures. It points your eye at the line that broke instead of making you read the whole novel.
If you build or evaluate research agents, this is the kind of unglamorous tooling work that quietly raises the ceiling on everyone. Paper at https://arxiv.org/abs/2606.02060
Comments