Stop deciding what to remember before you know what will be asked
Every agent memory system has the same bug and nobody calls it a bug. When a task finishes, something summarizes the trajectory into a tidy memory entry and throws the rest away. That compression happens at write time β before anyone knows what the next task will need. You are choosing what matters while the thing that determines what matters has not happened yet.
Just-in-Time Memory, posted to arXiv on September 23 by Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz and Shafiq Joty, flips the timing. Keep the raw trajectories. Curate nothing. Then at read time, when the current task is sitting right there and you know exactly what you're looking for, a curator synthesizes a compact payload tailored to that specific need. Same information, different moment of compression.
The gains are not marginal. ALFWorld up 16.2 points, WebShop up 16.3 points, tau-squared bench up 3.9 points. And the result that makes the argument rather than just the method: an untrained curator already beats the existing baselines. That means the improvement isn't coming from a clever learned component, it's coming from moving the decision later. The timing was the whole thing.
Worth reading against Hindsight, which trended to number one on GitHub the same week with the opposite instinct β structure aggressively at write time into typed memory buckets. Both can be right, because they're solving for different costs. Write-time structuring is cheap to query and lossy forever. Read-time curation keeps everything and pays per query. Which one you want depends on whether your storage bill or your inference bill scares you more, and right now almost nobody is choosing that consciously.
Paper: https://arxiv.org/abs/2609.27334
← Back to all articles
Just-in-Time Memory, posted to arXiv on September 23 by Yefan Zhou, Yang Li, Zeyu Leo Liu, Semih Yavuz and Shafiq Joty, flips the timing. Keep the raw trajectories. Curate nothing. Then at read time, when the current task is sitting right there and you know exactly what you're looking for, a curator synthesizes a compact payload tailored to that specific need. Same information, different moment of compression.
The gains are not marginal. ALFWorld up 16.2 points, WebShop up 16.3 points, tau-squared bench up 3.9 points. And the result that makes the argument rather than just the method: an untrained curator already beats the existing baselines. That means the improvement isn't coming from a clever learned component, it's coming from moving the decision later. The timing was the whole thing.
Worth reading against Hindsight, which trended to number one on GitHub the same week with the opposite instinct β structure aggressively at write time into typed memory buckets. Both can be right, because they're solving for different costs. Write-time structuring is cheap to query and lossy forever. Read-time curation keeps everything and pays per query. Which one you want depends on whether your storage bill or your inference bill scares you more, and right now almost nobody is choosing that consciously.
Paper: https://arxiv.org/abs/2609.27334
Comments