billion-context lets the model fold its own memory, if you let a proxy read everything
Most coding agents handle a full context window the same way: wait until it's almost full, then summarize everything at once and hope nothing important got lost. billion-context, an MIT-licensed project trending on GitHub this weekend (460 stars on Sunday, about 5,000 total), hands that decision to the model instead. Version 0.1.213 shipped Nov 7 at 23:25 UTC.
It sits between your agent and the Anthropic or OpenAI API and injects four tools into the conversation: compress folds a range of messages into a detailed summary, decompress brings the original back when exact details matter again, search_context does keyword search over summaries and visible messages, and acp_status shows how full the window is and what can still be folded. The model calls them itself as the context grows. Because summaries are written in small ranges and the prefix stays intact, the author says a healthy session keeps a 95-97% prompt-cache hit rate, with compression costing 2% or less.
The claims are big and all the author's own: "5x fewer tokens", sessions that run for days, and a 4.5-month study with 174,327 model calls and 18.76 billion input tokens with zero context-window overflows on 204,800-token models. The repo includes a paper and a 1,500-item benchmark run against LongMemEval-S, ContextBench and MemoryAgentBench, but it's the author's run in the author's repo. Treat it as a strong claim, not a verdict.
The support list is the striking part: Claude Code, Codex, OpenCode, Pi, Gemini CLI, Kimi, Qwen Code, Copilot CLI, Kiro, Zed, Goose and more. For the closed binaries with no plugin hook, the way in is "cert-MITM": you install a local certificate so the proxy can decrypt and rewrite your model traffic. That works, and it's also a full trust boundary. Everything your agent sends, code and secrets included, passes through it in the clear.
The direction is right. Context management is moving out of individual harnesses into a shared layer, and reversible, cache-friendly compression beats a single big summary. Just read the code before you hand it your TLS.
Source: https://github.com/ranxianglei/billion-context
← Back to all articles
It sits between your agent and the Anthropic or OpenAI API and injects four tools into the conversation: compress folds a range of messages into a detailed summary, decompress brings the original back when exact details matter again, search_context does keyword search over summaries and visible messages, and acp_status shows how full the window is and what can still be folded. The model calls them itself as the context grows. Because summaries are written in small ranges and the prefix stays intact, the author says a healthy session keeps a 95-97% prompt-cache hit rate, with compression costing 2% or less.
The claims are big and all the author's own: "5x fewer tokens", sessions that run for days, and a 4.5-month study with 174,327 model calls and 18.76 billion input tokens with zero context-window overflows on 204,800-token models. The repo includes a paper and a 1,500-item benchmark run against LongMemEval-S, ContextBench and MemoryAgentBench, but it's the author's run in the author's repo. Treat it as a strong claim, not a verdict.
The support list is the striking part: Claude Code, Codex, OpenCode, Pi, Gemini CLI, Kimi, Qwen Code, Copilot CLI, Kiro, Zed, Goose and more. For the closed binaries with no plugin hook, the way in is "cert-MITM": you install a local certificate so the proxy can decrypt and rewrite your model traffic. That works, and it's also a full trust boundary. Everything your agent sends, code and secrets included, passes through it in the clear.
The direction is right. Context management is moving out of individual harnesses into a shared layer, and reversible, cache-friendly compression beats a single big summary. Just read the code before you hand it your TLS.
Source: https://github.com/ranxianglei/billion-context
Comments