Agent memory benchmarks are all two-person conversations. Real life is a group chat.
Look at where agent memory gets evaluated and it's almost always one user talking to one assistant. Then look at where agents actually get deployed — Slack channels, team meetings, customer threads with three people from two companies — and the problem changes shape. It isn't only what was said. It's who said it, whether they're the one who gets to decide, and what the group agreed to versus what one person floated.
SpeakerMem-R1, top of the HuggingFace papers board today with 77 upvotes, is built for that. Dual-track memory: speaker-labeled verbatim messages on one side, derived state on the other, and the derived state is organized into two views — person-level and group-level. So "what does Wei want" and "what did this team decide" are separate lookups rather than the same pile of text filtered by hope. Training uses speaker-conditioned GRPO plus something they call SpeakerLevenshtein, aimed squarely at attribution errors, which is the characteristic way multi-party memory fails.
Numbers: 62.33% on the public EverMemBench leaderboard, which they report as best-reported, 69.2% on SocialMemBench, 70.85% on LoCoMo across 1,986 questions, 47.9% on GroupMemBench. That last one is the honest number and the interesting one — group-level memory is sitting under 50%, meaning on the specific thing this paper exists to fix, the field is still closer to a coin flip than to solved. A controlled RL evaluation moved mean accuracy from 57.38% to 68.20% across 305 questions.
It runs locally, which matters more here than usual, because the whole premise is feeding it everything your team says. Code is on GitHub via the project page.
Paper: https://arxiv.org/abs/2609.26780
← Back to all articles
SpeakerMem-R1, top of the HuggingFace papers board today with 77 upvotes, is built for that. Dual-track memory: speaker-labeled verbatim messages on one side, derived state on the other, and the derived state is organized into two views — person-level and group-level. So "what does Wei want" and "what did this team decide" are separate lookups rather than the same pile of text filtered by hope. Training uses speaker-conditioned GRPO plus something they call SpeakerLevenshtein, aimed squarely at attribution errors, which is the characteristic way multi-party memory fails.
Numbers: 62.33% on the public EverMemBench leaderboard, which they report as best-reported, 69.2% on SocialMemBench, 70.85% on LoCoMo across 1,986 questions, 47.9% on GroupMemBench. That last one is the honest number and the interesting one — group-level memory is sitting under 50%, meaning on the specific thing this paper exists to fix, the field is still closer to a coin flip than to solved. A controlled RL evaluation moved mean accuracy from 57.38% to 68.20% across 305 questions.
It runs locally, which matters more here than usual, because the whole premise is feeding it everything your team says. Code is on GitHub via the project page.
Paper: https://arxiv.org/abs/2609.26780
Comments