Gemma 4 26B, Running in 2GB of RAM, on an 8GB MacBook Air
turbo-fieldfare shot to the top of Hacker News with a claim that sounds impossible: it runs Gemma 4 26B-A4B in about 2GB of RAM, on Apple Silicon, including the 8GB base MacBook Air. The full model is 14.3GB. So how do you fit a 14GB brain into a 2GB room?
You don't load it all. Gemma 4 is a mixture-of-experts model, meaning any single token only actually uses a small slice of the weights. turbo-fieldfare keeps the shared 1.35GB core and the KV cache resident in memory, and streams the individual experts off the SSD on demand as generation needs them, with a 16-slot cache that keeps the hot ones around. Add custom Metal kernels for the 4-bit quantized math and you get 5 to 6 tokens per second on an M2 Air, which is genuinely usable for a chat or an agent loop. It's a native Swift and Metal runtime, Apache 2.0, and it ships a Mac app, a CLI, and an OpenAI-compatible loopback server with function-tool support.
That last detail is the one that matters here. This isn't a chatbot demo, it's a local inference engine with tool calling, which means you can point an agent at localhost and run the whole loop on a laptop that costs nothing extra, offline, with your data never leaving the machine. The story of local AI for two years has been buy more VRAM. Expert streaming flips it: the bottleneck moves from how much memory you have to how fast your SSD reads, and SSDs are cheap. A frontier-class MoE on the cheapest Mac Apple sells is the kind of thing that quietly resets what an independent builder can do. Repo is github.com/drumih/turbo-fieldfare.
← Back to all articles
You don't load it all. Gemma 4 is a mixture-of-experts model, meaning any single token only actually uses a small slice of the weights. turbo-fieldfare keeps the shared 1.35GB core and the KV cache resident in memory, and streams the individual experts off the SSD on demand as generation needs them, with a 16-slot cache that keeps the hot ones around. Add custom Metal kernels for the 4-bit quantized math and you get 5 to 6 tokens per second on an M2 Air, which is genuinely usable for a chat or an agent loop. It's a native Swift and Metal runtime, Apache 2.0, and it ships a Mac app, a CLI, and an OpenAI-compatible loopback server with function-tool support.
That last detail is the one that matters here. This isn't a chatbot demo, it's a local inference engine with tool calling, which means you can point an agent at localhost and run the whole loop on a laptop that costs nothing extra, offline, with your data never leaving the machine. The story of local AI for two years has been buy more VRAM. Expert streaming flips it: the bottleneck moves from how much memory you have to how fast your SSD reads, and SSDs are cheap. A frontier-class MoE on the cheapest Mac Apple sells is the kind of thing that quietly resets what an independent builder can do. Repo is github.com/drumih/turbo-fieldfare.
Comments