Qwen3.8-Flash-Next opens a Qwen4 preview as NVIDIA validates long-context inference on GB300

Alibaba’s Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal mixture-of-experts model that serves as an early preview of the architecture planned for the Qwen4 family. The August 26 announcement is notable because it is not only another model checkpoint. Qwen is exposing architectural changes before the next full generation arrives, giving developers, researchers and infrastructure teams a chance to test the design choices that may shape its future production models.

The model combines a 125 billion-parameter main network with an additional 51 billion parameters in N-gram embeddings and activates about 6 billion parameters per token. Qwen says the model natively supports a 262,144-token context window and can be extended to one million tokens with YaRN. That makes the release directly relevant to agentic coding, document analysis and enterprise workflows where useful context can include repositories, contracts, logs, emails and tool traces rather than a short prompt.

The core architectural change is a hybrid of Gated DeltaNet and Qwen Sparse Attention. Three out of every four layers use Gated DeltaNet to compress historical context into a fixed-size recurrent state, while the remaining attention layer retrieves information across the full context. Qwen Sparse Attention adds a lightweight indexer that groups context into micro-blocks and selects the most important regions instead of scanning every token with the same cost. Qwen says this approach reduces both attention computation and indexing overhead, especially when context grows toward the million-token range.

Qwen’s own benchmarks claim that at one million tokens, Qwen Sparse Attention delivers up to 7.6 times faster prefill and 4.9 times faster decoding than full attention. In a serving setup with a 90 percent prefix-cache hit rate, Qwen3.8-Flash-Next achieved 8.6 times the prefill throughput of Qwen3.7-Plus. The team also says training required about one-ninth the cost of Qwen3.7-Plus while improving capability in coding and office tasks. These claims will need broader independent testing, but they point at the industry’s most important efficiency problem: long-context models are only useful if they can be served at tolerable latency and cost.

NVIDIA moved quickly to position the model inside its accelerated computing stack. In a separate technical post published the same day, NVIDIA said it is providing best-effort Day 0 functional support through SGLang, vLLM and TensorRT-LLM, with validation on the GB300 NVL72 platform and post-training recipes through NeMo AutoModel and NeMo RL. NVIDIA reported that running Qwen3.8-Flash-Next on GB300 NVL72 delivered more than 16,000 tokens per second per GPU and more than 200 tokens per second per user.

That hardware validation matters because open weights alone do not guarantee practical adoption. Developers need model files, inference recipes, fine-tuning paths and production serving guidance. Qwen says weights are available on Hugging Face and ModelScope, while the managed production version is served as Qwen3.8-Flash on QwenCloud. The company listed pricing of 0.16 dollars per million input tokens and 0.47 dollars per million output tokens, with API access described as coming shortly after the post.

The release shows where the open-model race is heading. The contest is no longer only about benchmark scores at ordinary context length. It is about whether a model can carry large working memory, reason over long artifacts and run cheaply enough for agents that make repeated calls throughout a task. Qwen3.8-Flash-Next gives developers a new architecture to inspect, and NVIDIA’s immediate support signals that infrastructure vendors see long-context inference as a production workload rather than a research novelty.