Qwen3.8-27B at 1500 Tokens/s: Cerebras Puts the Best Small Model on Fast-Forward
Cerebras added Qwen3.8-27B to its public inference catalog at roughly 1,500 tokens per second with up to 128k context, and Hacker News pushed it to 346 points. The pairing matters more than either half: this is arguably the best small open model of the year — 73 on Terminal-Bench, 84.3 on OSWorld, numbers we covered when it dropped (https://clauday.com/article/5ca4092d-2ab9-4e01-9cbc-c0dcb720c5f2) — now running at wafer-scale speed.
The engineering details hold up under inspection, which hasn't always been true of fast-inference claims. Cerebras says the endpoints serve original, unpruned weights; quantization is selective and storage-only, with sensitive layers in full precision and activations, attention, and KV cache untouched. So the Terminal-Bench score you read about is the model you're actually calling.
Why speed is the story: for chat, 1,500 tokens/s is a party trick. For agents, it's the difference between a 40-minute autonomous run and a 4-minute one. Agent loops are serial — think, call tool, read result, think again — so latency compounds with every step. A 27B model that scores like a mid-tier frontier model and streams this fast is a better agent substrate than a smarter model that makes you wait.
Model catalog at https://inference-docs.cerebras.ai/models/overview.
← Back to all articles
The engineering details hold up under inspection, which hasn't always been true of fast-inference claims. Cerebras says the endpoints serve original, unpruned weights; quantization is selective and storage-only, with sensitive layers in full precision and activations, attention, and KV cache untouched. So the Terminal-Bench score you read about is the model you're actually calling.
Why speed is the story: for chat, 1,500 tokens/s is a party trick. For agents, it's the difference between a 40-minute autonomous run and a 4-minute one. Agent loops are serial — think, call tool, read result, think again — so latency compounds with every step. A 27B model that scores like a mid-tier frontier model and streams this fast is a better agent substrate than a smarter model that makes you wait.
Model catalog at https://inference-docs.cerebras.ai/models/overview.
Comments