Needle 3 is 8MB and it out-calls tools against models ten times its size
Cactus Compute shipped Needle 3, an automation foundation model for tiny devices, and the size numbers are the pitch: 8 to 29 MB on disk, 29 to 121 million parameters, weights at CQ2-bit. It beats models ten times its size on mobile tool calls and matches two to three times bigger models on structured extraction. Fine-tuned on Cactus's platform, a four-layer Needle 3 starting at 29M parameters passes DeepSeek V4 Flash. Show HN post is at https://cactuscompute.com/needle, weights and per-platform engines on Hugging Face at Cactus-Compute/needle3, code at https://github.com/cactus-compute/needle.
The architectural trick is intelligence laddering. One binary contains sub-networks from 2 to 20 layers, so the same artifact serves a microcontroller and a phone and you pick the depth at deploy time instead of shipping five models. Underneath it is a Laddered Simple Attention Network with a Monarch Hadamard MLP replacing the FFN, GQA attention with causal conv taps, an engram n-gram memory read by gather, and multi-lane hyper-connections. The claim is the 121M model does the arithmetic of a 50M one. Every generated token is constrained by a byte-level grammar compiled from your schema, which is why extraction works at this size at all: the model is not being trusted to produce valid JSON, it is structurally prevented from producing anything else.
It does three things and only three. Tool calling, which means picking the function and filling the arguments from a user request. Structured extraction into typed JSON fields. Text embeddings for local semantic search. That narrowness is the whole reason the numbers work, and it is a better argument about where agents are going than most of the frontier-model discourse. The part of an agent that decides which tool to call and with what arguments does not need a frontier model. It needs to be fast, local, cheap, and constrained.
Fine-tuning on a specific dataset moved performance 18 to 36 percentage points, which is enormous and also the honest caveat: out of the box it is good, task-tuned it is the thing that beats DeepSeek V4 Flash. Needle is Apache 2.0 while the Cactus Engine is source-available under its own license, so check which piece you are depending on. Prebuilt engines exist for macOS, Linux, Windows, Android, iOS and WebAssembly.
← Back to all articles
The architectural trick is intelligence laddering. One binary contains sub-networks from 2 to 20 layers, so the same artifact serves a microcontroller and a phone and you pick the depth at deploy time instead of shipping five models. Underneath it is a Laddered Simple Attention Network with a Monarch Hadamard MLP replacing the FFN, GQA attention with causal conv taps, an engram n-gram memory read by gather, and multi-lane hyper-connections. The claim is the 121M model does the arithmetic of a 50M one. Every generated token is constrained by a byte-level grammar compiled from your schema, which is why extraction works at this size at all: the model is not being trusted to produce valid JSON, it is structurally prevented from producing anything else.
It does three things and only three. Tool calling, which means picking the function and filling the arguments from a user request. Structured extraction into typed JSON fields. Text embeddings for local semantic search. That narrowness is the whole reason the numbers work, and it is a better argument about where agents are going than most of the frontier-model discourse. The part of an agent that decides which tool to call and with what arguments does not need a frontier model. It needs to be fast, local, cheap, and constrained.
Fine-tuning on a specific dataset moved performance 18 to 36 percentage points, which is enormous and also the honest caveat: out of the box it is good, task-tuned it is the thing that beats DeepSeek V4 Flash. Needle is Apache 2.0 while the Cactus Engine is source-available under its own license, so check which piece you are depending on. Prebuilt engines exist for macOS, Linux, Windows, Android, iOS and WebAssembly.
Comments