antirez Built a C Inference Engine for DeepSeek V4
Salvatore Sanfilippo, better known as antirez, the guy who wrote Redis, has a new obsession called ds4 (DwarfStar): a local inference engine for DeepSeek V4 Flash and PRO plus GLM 5.2, hand-written in C for Metal, CUDA and ROCm. It's sitting at 20.3k stars and climbing GitHub trending days after DeepSeek V4 Flash dropped its open weights.
The engineering is classic antirez. Asymmetric quantization that squeezes only the routed MoE experts down to 2 bits while leaving attention and shared components untouched. SSD streaming for models bigger than your RAM. Tensor parallelism across multiple Macs over Thunderbolt. Speculative decoding. And a built-in coding agent with persistent KV cache, so sessions save to disk and resume where they left off. On an M3 Max with 128GB he reports around 27 tokens per second of generation from a 284B-parameter model. That's a real coding assistant running on a laptop with no API key.
The bigger signal is what happens when a genuinely open frontier model meets the systems-programming crowd. DeepSeek shipped V4 Flash open on July 31; within days one of the most respected C programmers alive is optimizing it for consumer hardware, for free. Closed labs pay infra teams for this. Open labs get it as a gift. That's the compounding advantage of open weights that no pricing page captures.
Repo: https://github.com/antirez/ds4
← Back to all articles
The engineering is classic antirez. Asymmetric quantization that squeezes only the routed MoE experts down to 2 bits while leaving attention and shared components untouched. SSD streaming for models bigger than your RAM. Tensor parallelism across multiple Macs over Thunderbolt. Speculative decoding. And a built-in coding agent with persistent KV cache, so sessions save to disk and resume where they left off. On an M3 Max with 128GB he reports around 27 tokens per second of generation from a 284B-parameter model. That's a real coding assistant running on a laptop with no API key.
The bigger signal is what happens when a genuinely open frontier model meets the systems-programming crowd. DeepSeek shipped V4 Flash open on July 31; within days one of the most respected C programmers alive is optimizing it for consumer hardware, for free. Closed labs pay infra teams for this. Open labs get it as a gift. That's the compounding advantage of open weights that no pricing page captures.
Repo: https://github.com/antirez/ds4
Comments