NVIDIA's Tri-Mode Diffusion LLM: One Model, No Draft Model Needed

NVIDIA Releases Nemotron-Labs-Diffusion Open Weights on 10 July 2026

NVIDIA released open weights for Nemotron-Labs-Diffusion on 10 July 2026 — a language model family spanning 3B, 8B, and 14B parameters that can switch between three decoding modes at inference time without any weight changes. The weights are available on Hugging Face under a commercial licence, with base, instruction-tuned, and vision-language variants at each scale. Inference support is live in vLLM and SGLang.

The release addresses one of the practical bottlenecks in deploying large language models at production scale: the throughput ceiling imposed by autoregressive token-by-token generation. Autoregressive models are computationally serial — each token depends on all preceding tokens, which means the GPU must perform one forward pass per output token, and throughput scales poorly with sequence length. Nemotron-Labs-Diffusion introduces two additional modes — diffusion parallel decoding and self-speculation — that bypass this serial constraint without requiring a second, separate model.

The Three Decoding Modes

The first mode is standard autoregressive decoding, which all existing language model inference frameworks already support. The second mode is diffusion-based parallel decoding, in which the model generates an entire block of output tokens simultaneously rather than one at a time. Parallel decoding trades some quality for throughput: generating many tokens at once means each token is less informed by the others in the block.

The third mode — self-speculation — is where the architecture delivers its most significant throughput improvement. In conventional speculative decoding, a small, fast draft model generates candidate tokens and a larger target model verifies them in parallel, accepting the longest valid prefix. The overhead of running two separate models has historically limited the efficiency gains from this approach. Nemotron-Labs-Diffusion eliminates the separate draft model entirely: it uses its own diffusion attention pathway to generate candidate tokens in parallel, then uses its autoregressive attention pathway to verify them. Both pathways share the same weights and the same KV cache. The total overhead is the cost of running one model, not two.

The Performance Numbers

On NVIDIA's SPEED-Bench evaluation — measured at batch size 1 and concurrency 1 on a GB200 GPU running SGLang — the 8B instruction-tuned model with the LoRA adapter averaged 6.82 accepted tokens per draft step in self-speculation mode. Eagle3, the leading external speculative decoding method, averaged 2.75 accepted tokens per step on the same evaluation. That is a 2.5x improvement in the number of tokens accepted per verification step.

The practical throughput gain on SPEED-Bench translates to approximately 4x higher generation throughput for the 8B model when self-speculation is used compared to standard autoregressive decoding on a GB200 GPU with SGLang. This means roughly 4x more tokens generated per second per GPU without any drop in output quality relative to the autoregressive baseline. A speed-of-light analysis in the research demonstrates that the diffusion mode has long-term potential for up to 76.5 per cent more tokens per forward pass than self-speculation under an optimal sampler.

The Architecture Behind Self-Speculation

The research insight behind Nemotron-Labs-Diffusion is that the same model can be trained on both autoregressive and masked-diffusion objectives simultaneously. During pretraining, the model learns to predict tokens causally and to denoise masked token sequences. The two objectives require different attention patterns — causal unidirectional for autoregressive, bidirectional for diffusion — and the model applies the appropriate attention mask for each mode at inference time without changing any weights.

This joint training is what makes self-speculation work with a single set of weights. The autoregressive verification pathway and the diffusion drafting pathway share the same representation space because they were trained together. In self-speculation: the diffusion path generates a block of candidate tokens in parallel, then the autoregressive path runs a second forward pass over those candidates and accepts the longest contiguous prefix that matches its own predictions. Because both paths are the same model with shared weights, the verification step is genuine.

What Nemotron-Labs-Diffusion Means for Indian Engineering and AI Teams

The practical question for engineering teams is whether Nemotron-Labs-Diffusion changes the economics of open-weight inference deployments. For teams running self-hosted inference on GPU infrastructure — on-premise or on cloud GPU rentals such as Lambda Labs or the emerging Indian GPU cloud providers under the IndiaAI Mission compute programme — a 4x throughput improvement per GPU directly reduces the number of GPUs needed to serve a given request volume. At current Indian GPU cloud spot rates, a 4x throughput improvement translates to roughly a 75 per cent reduction in per-token inference cost on the same hardware.

The model family's size range makes it practical for a wider deployment scenario than a single large open-weight model. The 3B model fits comfortably on a single consumer-grade or entry-level server GPU, the 8B model runs on a single A100 80GB or H100 80GB, and the 14B model requires either a large single GPU or a small two-GPU setup. Teams that previously needed two H100s to run a comparable open-weight model may be able to serve equivalent throughput with one, halving the GPU cost.

For Indian fintech, healthtech, and regulated-industry teams considering open-weight models for data residency and audit compliance, the commercial licence and open-weight availability of Nemotron-Labs-Diffusion means the model can be deployed on Indian infrastructure without an API dependency on a foreign provider.

The Bottom Line

NVIDIA released Nemotron-Labs-Diffusion open weights on 10 July 2026 — a 3B, 8B, and 14B tri-mode language model supporting autoregressive decoding, diffusion parallel decoding, and self-speculation from the same weights. In self-speculation mode, the 8B instruction-tuned model achieves 6.82 accepted tokens per draft step versus Eagle3's 2.75, and delivers approximately 4x higher throughput than standard autoregressive decoding on a GB200 GPU with SGLang. No separate draft model is required. Inference support is available in vLLM and SGLang, and the model ships under a commercial licence on Hugging Face with base, instruction-tuned, and vision-language variants at all three parameter scales. For Indian engineering teams managing the cost of open-weight inference at production scale, the throughput improvement directly reduces the GPU requirement for a given request volume.