Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal MoE model that also serves as an early preview of the architecture planned for Qwen4.
Key characteristics
- 125 billion parameters in the main model;
- 51 billion additional N-gram embedding parameters;
- about 6 billion active parameters per token;
- up to 256K native context;
- support for coding agents, computer use, and vision.
The core idea is not to run the entire model for every token. Only a subset of parameters is activated, while the remaining capacity acts as additional stored knowledge.
Architecture changes
Qwen combines Gated DeltaNet, Qwen Sparse Attention, N-gram embeddings, and Multi-Token Prediction. Gated DeltaNet compresses history into a compact state, while Sparse Attention searches relevant information in micro-blocks rather than scanning every token.
N-gram embeddings provide a fast lookup for common patterns. Multi-Token Prediction can improve generation speed by proposing several next tokens for verification.
Results and limitations
In Qwen’s published evaluations, Flash-Next outperformed Qwen3.8-27B on several tasks, including JobBench, DeepSWE 1.1, and SWE-bench Multilingual.
However, 6 billion active parameters do not mean the full model fits in memory like a standard 7B model. The practical setup depends on quantization, RAM, SSD, VRAM, and inference-engine support.
A more accurate description is not “125B running like 7B,” but “a large-capacity model trying to keep per-token compute closer to smaller models.”
Why it matters
Qwen3.8-Flash-Next is relevant for coding agents, large-document analysis, repository work, and computer use. It points toward a future in which local AI does not only shrink models, but activates only the knowledge required at a given moment.
Official repository: QwenLM/Qwen3.8-Flash-Next. Hugging Face: Qwen3.8-Flash-Next.