Trace2World and the Slovak web audit: check against the source, not yourself
Two papers from Friday's arXiv batch land on the same rule for building agent systems: the check has to come from outside the thing being checked, and once something is found, turn it into a deterministic test.
Trace2World builds training environments for GUI agents out of real user interaction traces. From 1,570 traces it builds 50 app environments, and on 200 workflows across ten apps it reproduces 91.1% end to end, against 59.8% for the strongest task-local baseline. The number that matters most is about validation. When the simulated app is validated against the real source traces, 94.0% of the training gains carry over to real devices. When the simulator validates itself, only 54.7% do. Training in the reconstructed worlds added 20.2 points on their benchmark, 5.4 to 16.2 points on four outside GUI benchmarks, and 16.3 points on 100 real-device tasks. In short, a simulator that grades its own homework loses almost half the value.
The second paper turns an agent loose on Slovak FineWeb2, a public pretraining corpus. Three runs costing $2.21 to $2.84 each produced seven findings, and the largest was a machine-generated content farm of 1,408,437 documents, 5.32% of the corpus, that had passed every filter in the pipeline. Blinded human labels backed six of the seven. The findings were then compiled into a test suite that scans the whole corpus in 0.7 seconds, compared with $12k to $90k for one pass of LLM classification. The authors also report their negative result plainly: the best static sampling baseline validated more selectors than any agent run, at under half the cost.
That second pattern is the one to steal. Use the agent once, for discovery, where its flexibility is worth paying for. Have a human confirm what it found. Then freeze it into code that runs for free forever. It's Drew Breunig's "knowing when to be deterministic" turned into a pipeline, and it's the opposite of putting an LLM call in every step and paying for it every time.
Trace2World: https://arxiv.org/abs/2610.30792 Agentic audit of Slovak FineWeb2: https://arxiv.org/abs/2610.31636
← Back to all articles
Trace2World builds training environments for GUI agents out of real user interaction traces. From 1,570 traces it builds 50 app environments, and on 200 workflows across ten apps it reproduces 91.1% end to end, against 59.8% for the strongest task-local baseline. The number that matters most is about validation. When the simulated app is validated against the real source traces, 94.0% of the training gains carry over to real devices. When the simulator validates itself, only 54.7% do. Training in the reconstructed worlds added 20.2 points on their benchmark, 5.4 to 16.2 points on four outside GUI benchmarks, and 16.3 points on 100 real-device tasks. In short, a simulator that grades its own homework loses almost half the value.
The second paper turns an agent loose on Slovak FineWeb2, a public pretraining corpus. Three runs costing $2.21 to $2.84 each produced seven findings, and the largest was a machine-generated content farm of 1,408,437 documents, 5.32% of the corpus, that had passed every filter in the pipeline. Blinded human labels backed six of the seven. The findings were then compiled into a test suite that scans the whole corpus in 0.7 seconds, compared with $12k to $90k for one pass of LLM classification. The authors also report their negative result plainly: the best static sampling baseline validated more selectors than any agent run, at under half the cost.
That second pattern is the one to steal. Use the agent once, for discovery, where its flexibility is worth paying for. Have a human confirm what it found. Then freeze it into code that runs for free forever. It's Drew Breunig's "knowing when to be deterministic" turned into a pipeline, and it's the opposite of putting an LLM call in every step and paying for it every time.
Trace2World: https://arxiv.org/abs/2610.30792 Agentic audit of Slovak FineWeb2: https://arxiv.org/abs/2610.31636
Comments