DeepSeek V4 Flash on One AMD GPU: the Ecosystem Wave Hits Hardware
A repo hit 344 points on HN today for running DeepSeek-V4-Flash-0731, all 304 billion parameters of it, in production on a single AMD MI300X. The numbers: 168.6 tokens per second single-stream decode, around 8K tokens per second prefill, 542 tokens per second aggregate across 8 concurrent streams, 256K context validated against an architecture that supports 1M.
Getting there took real engineering, not a config file. MI300X speaks AMD's FNUZ FP8 format instead of the OCP standard, which needed custom cache writers. The author tuned AITER GEMM kernel tables for the gfx942 architecture, fixed a MoE routing bug where bitmatrix padding was silently corrupting tool calls, split the KV cache across 20GB of GPU and 96GB of CPU, and wired in speculative decoding with a DSpark-7 draft model. All of it documented, Apache 2.0.
Count the week: ds4, antirez's C inference engine. DeepSeek-Reasonix, a Go coding agent built around prefix-cache economics. AirLLM streaming frontier models on gaming cards. Cloudflare serving Kimi and GLM at the edge. Now a production recipe for the one GPU vendor that is not Nvidia. Five days after DeepSeek opened V4 Flash, it has an inference engine, a harness, and a hardware stack that Nvidia does not tax. Open weights do not just get downloaded, they grow infrastructure.
Repo at github.com/ryanzhou/deepseek-v4-flash-mi300x.
← Back to all articles
Getting there took real engineering, not a config file. MI300X speaks AMD's FNUZ FP8 format instead of the OCP standard, which needed custom cache writers. The author tuned AITER GEMM kernel tables for the gfx942 architecture, fixed a MoE routing bug where bitmatrix padding was silently corrupting tool calls, split the KV cache across 20GB of GPU and 96GB of CPU, and wired in speculative decoding with a DSpark-7 draft model. All of it documented, Apache 2.0.
Count the week: ds4, antirez's C inference engine. DeepSeek-Reasonix, a Go coding agent built around prefix-cache economics. AirLLM streaming frontier models on gaming cards. Cloudflare serving Kimi and GLM at the edge. Now a production recipe for the one GPU vendor that is not Nvidia. Five days after DeepSeek opened V4 Flash, it has an inference engine, a harness, and a hardware stack that Nvidia does not tax. Open weights do not just get downloaded, they grow infrastructure.
Repo at github.com/ryanzhou/deepseek-v4-flash-mi300x.
Comments