Bonsai 27B Runs a 27B Model on Your iPhone
PrismML just squeezed a 27B-parameter multimodal model into 3.9GB and ran it on an iPhone 17 Pro. Eleven tokens a second. On an RTX 5090 the 1-bit version hits 163 tok/s, on an M5 Max it does 87. The whole thing is Apache 2.0 and downloadable today.
The trick is extreme quantization. Bonsai 27B is built on Qwen3.6 27B and ships in two flavors: a ternary variant where every weight is crushed down to -1, 0 or +1 (about 1.71 effective bits) and a 1-bit binary variant at roughly 1.1 bits. PrismML claims ternary keeps more than 95 percent of full-precision benchmark performance and the 1-bit keeps more than 90. Weights are on Hugging Face as prism-ml/Bonsai-27B-mlx-1bit and prism-ml/Bonsai-27B-gguf.
Now the part the press release will not tell you. On Hacker News, where this pulled 266 points in a few hours, the people who actually ran it are not celebrating. One developer got nothing but exclamation marks out of it on Android. Another tested the 4-bit variant and got 16.75 on wikitext but a flat zero on gsm8k. And the most damning comment for anyone reading this site: the capability that degrades first under aggressive quantization is tool calling. That is the whole ballgame for agents. A model that chats fine on your phone but drops a function call every twelfth turn is not a local agent, it is a demo.
The direction still matters. 27B-class weights in under 4GB is what a phone-resident agent eventually looks like, and it lands the same week Ollama took 65 million dollars to keep local inference the default. The rent-versus-run fight was never about whether small models can talk. It is about whether they can act. Watch the tool-calling numbers, not the token throughput.
https://prismml.com/news/prismml-releases-bonsai-27b
← Back to all articles
The trick is extreme quantization. Bonsai 27B is built on Qwen3.6 27B and ships in two flavors: a ternary variant where every weight is crushed down to -1, 0 or +1 (about 1.71 effective bits) and a 1-bit binary variant at roughly 1.1 bits. PrismML claims ternary keeps more than 95 percent of full-precision benchmark performance and the 1-bit keeps more than 90. Weights are on Hugging Face as prism-ml/Bonsai-27B-mlx-1bit and prism-ml/Bonsai-27B-gguf.
Now the part the press release will not tell you. On Hacker News, where this pulled 266 points in a few hours, the people who actually ran it are not celebrating. One developer got nothing but exclamation marks out of it on Android. Another tested the 4-bit variant and got 16.75 on wikitext but a flat zero on gsm8k. And the most damning comment for anyone reading this site: the capability that degrades first under aggressive quantization is tool calling. That is the whole ballgame for agents. A model that chats fine on your phone but drops a function call every twelfth turn is not a local agent, it is a demo.
The direction still matters. 27B-class weights in under 4GB is what a phone-resident agent eventually looks like, and it lands the same week Ollama took 65 million dollars to keep local inference the default. The rent-versus-run fight was never about whether small models can talk. It is about whether they can act. Watch the tool-calling numbers, not the token throughput.
https://prismml.com/news/prismml-releases-bonsai-27b
Comments