Qwen Ships the Qwen4 Architecture Early, on Purpose
Alibaba's Qwen team released Qwen3.8-Flash-Next as open weights, and the name undersells what it is: the first public model built on the architecture that will power Qwen4, shipped months ahead of the family so the ecosystem can get ready. Their stated reason is exactly that blunt. The architectural changes should be in the community's hands before Qwen4 arrives, so runtimes, quantizations, and applications work on day one.
The architecture is where it gets interesting. A 125B-parameter MoE with only 6B active per token, paired with a 51B n-gram embedding table and a 4B multi-token prediction module, and a new attention scheme that swaps in Qwen Sparse Attention alongside Gated DeltaNet. Native 262K context, natively multimodal. Those are unusual proportions, an embedding table nearly half the size of the backbone, and they signal where Qwen thinks the efficiency frontier is: spend parameters on memory-like lookup, keep the per-token compute tiny.
For a 6B-active model the numbers are legitimately startling, 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 84.5 on AndroidWorld. A year ago those were frontier-flagship scores. The production Qwen3.8-Flash will run $0.16 per million input tokens on their API, which continues the theme of the week, that the price floor for competent agentic coding keeps falling through the floor below it.
The meta-move deserves a beat. Publishing your next-generation architecture as open weights before the generation itself exists is something closed labs structurally cannot do, and it turns the community into a free integration team. When Qwen4 lands, every inference stack will already run it. That is the open-weights playbook maturing from we release models into we release roadmaps. Weights at huggingface.co/Qwen/Qwen3.8-Flash-Next.
← Back to all articles
The architecture is where it gets interesting. A 125B-parameter MoE with only 6B active per token, paired with a 51B n-gram embedding table and a 4B multi-token prediction module, and a new attention scheme that swaps in Qwen Sparse Attention alongside Gated DeltaNet. Native 262K context, natively multimodal. Those are unusual proportions, an embedding table nearly half the size of the backbone, and they signal where Qwen thinks the efficiency frontier is: spend parameters on memory-like lookup, keep the per-token compute tiny.
For a 6B-active model the numbers are legitimately startling, 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 84.5 on AndroidWorld. A year ago those were frontier-flagship scores. The production Qwen3.8-Flash will run $0.16 per million input tokens on their API, which continues the theme of the week, that the price floor for competent agentic coding keeps falling through the floor below it.
The meta-move deserves a beat. Publishing your next-generation architecture as open weights before the generation itself exists is something closed labs structurally cannot do, and it turns the community into a free integration team. When Qwen4 lands, every inference stack will already run it. That is the open-weights playbook maturing from we release models into we release roadmaps. Weights at huggingface.co/Qwen/Qwen3.8-Flash-Next.
Comments