LongStraw: everyone serves 1M context, almost nobody can train on it
Here's an embarrassing asymmetry in the agent stack: inference systems happily serve million-token contexts, but RL post-training — the thing that actually makes agents better at long tasks — tops out around 256K tokens. So an agent that accumulates a two-million-token trajectory of observations cannot learn from it end-to-end. Everyone measures long-horizon performance. Almost nobody can train for it.
LongStraw, the top paper on HuggingFace with 171 upvotes, is an execution stack that pushes GRPO-based RL past 2M tokens on a fixed GPU budget. The tricks are architecture-aware plumbing, not new math: evaluate the shared prompt without building autodiff graphs, keep only model-specific state, replay response branches sequentially so only one training graph is alive at a time. The result: 2.1M token positions trained on eight H20 GPUs with memory growing just 0.21GB per unit of group size, validated end-to-end through all 78 layers of GLM-5.2 on 32 H20s at full 2.1M-token prompt length.
Note the hardware. H20s are the export-control-compliant China chips — the "fixed GPU budget" in the title is a real constraint turned into a design principle, squeezing frontier-scale RL out of capped silicon.
Why it matters for the thesis: the long-horizon agent benchmarks everyone is chasing reward exactly the behavior you can only train with long-context RL. Whoever closes this train-serve gap gets agents that improve on their own long trajectories instead of just replaying them. Paper: arxiv.org/abs/2607.14952
← Back to all articles
LongStraw, the top paper on HuggingFace with 171 upvotes, is an execution stack that pushes GRPO-based RL past 2M tokens on a fixed GPU budget. The tricks are architecture-aware plumbing, not new math: evaluate the shared prompt without building autodiff graphs, keep only model-specific state, replay response branches sequentially so only one training graph is alive at a time. The result: 2.1M token positions trained on eight H20 GPUs with memory growing just 0.21GB per unit of group size, validated end-to-end through all 78 layers of GLM-5.2 on 32 H20s at full 2.1M-token prompt length.
Note the hardware. H20s are the export-control-compliant China chips — the "fixed GPU budget" in the title is a real constraint turned into a design principle, squeezing frontier-scale RL out of capped silicon.
Why it matters for the thesis: the long-horizon agent benchmarks everyone is chasing reward exactly the behavior you can only train with long-context RL. Whoever closes this train-serve gap gets agents that improve on their own long trajectories instead of just replaying them. Paper: arxiv.org/abs/2607.14952
Comments