EnvACE: Tencent Teaches Agents to Rehearse the World in Their Heads
The most expensive part of agent RL is not the GPU, it is the environment — real APIs, real sandboxes, real latency, real flakiness. EnvACE, a new paper from a Tencent-affiliated team (arXiv 2608.06197, 38 upvotes on HuggingFace Daily Papers), asks the obvious heretical question: what if the policy just learns to simulate the environment itself?
The method is called world rehearsal. During training the agent alternates between generating tool calls and generating the environment's responses to those calls, optimized end-to-end with task-success rewards. The world model is not a separate module bolted on the side; it gets internalized into the same policy. Across BFCL-v4, tau2-Bench, VitaBench and FinMCP-Bench, this beats the environment-scaling baselines — the approach where you just stand up more and more real environments. And there is a bonus at inference: the agent can rehearse privately in its head before touching the real world, gaining accuracy without extra external calls.
Why this matters: everyone building agent RL pipelines has hit the wall where environments, not compute, are the bottleneck — they are slow to build, expensive to run, and break constantly. If policies can amortize environment dynamics into their own weights, the scarce resource shifts from environment engineering back to data and reward design. Worth watching whether the internalized world model hallucinates conveniently optimistic responses at scale, which is the failure mode this whole research thread has to answer for.
Code at github.com/Within-yao/EnvACE, paper at arxiv.org/abs/2608.06197.
← Back to all articles
The method is called world rehearsal. During training the agent alternates between generating tool calls and generating the environment's responses to those calls, optimized end-to-end with task-success rewards. The world model is not a separate module bolted on the side; it gets internalized into the same policy. Across BFCL-v4, tau2-Bench, VitaBench and FinMCP-Bench, this beats the environment-scaling baselines — the approach where you just stand up more and more real environments. And there is a bonus at inference: the agent can rehearse privately in its head before touching the real world, gaining accuracy without extra external calls.
Why this matters: everyone building agent RL pipelines has hit the wall where environments, not compute, are the bottleneck — they are slow to build, expensive to run, and break constantly. If policies can amortize environment dynamics into their own weights, the scarce resource shifts from environment engineering back to data and reward design. Worth watching whether the internalized world model hallucinates conveniently optimistic responses at scale, which is the failure mode this whole research thread has to answer for.
Code at github.com/Within-yao/EnvACE, paper at arxiv.org/abs/2608.06197.
Comments