Red Hat AI Releases DFlash Checkpoints to Accelerate Inference for NVIDIA Nemotron Models

Red Hat AI has announced the release of DFlash speculator checkpoints for two of NVIDIA AI's largest open source models: Nemotron Ultra 550B and Nemotron Super 120B. The announcement was made via Red Hat AI's official X account on July 22, 2026. DFlash is a speculative decoding technique that uses a lightweight diffusion based draft model to generate blocks of tokens in parallel, while a verifier model validates all generated tokens in a single forward pass. According to Red Hat AI's post, the checkpoints were trained using the open source Speculators library from the vLLM Project and are released under the Apache 2.0 license. Both checkpoints have been validated on NVIDIA B200 GPUs.

To enable them in vLLM, users need to add a single flag: , spec model RedHatAI/NVIDIA Nemotron 3 Ultra 550B A55B speculator.dflash, spec tokens 7, spec method dflash . Data from Red Hat AI indicates that on math and reasoning tasks, an average of about 5 out of 7 draft tokens are accepted. On code tasks (HumanEval), an average of about 3.4 out of 7 draft tokens are accepted. These figures demonstrate DFlash's efficiency in generating valid tokens without requiring repeated regeneration. The Nemotron Ultra 550B model is a hybrid Mamba Transformer Mixture of Experts (MoE) model with 55 billion active parameters out of a total of 550 billion parameters. It uses Latent MoE techniques, includes MTP Layers, and was trained in NVFP4 format. NVIDIA released the model on June 4, 2026, through its research division.

The Nemotron Super 120B has 12 billion active parameters out of a total of 120 billion parameters. Speculative decoding techniques like DFlash accelerate inference by generating multiple tokens at once in a single step, rather than one at a time. This approach reduces the number of forward passes required, thereby lowering latency and increasing throughput. For large models like Nemotron Ultra, this acceleration is particularly significant for real time applications such as chatbots, AI assistants, or recommendation systems. Red Hat AI emphasized that these checkpoints are designed to make it easier for developers and researchers to adopt speculative decoding without training a draft model from scratch. With the Apache 2.0 license, users can freely modify and distribute the checkpoints.

This aligns with the open source trend in the AI ecosystem that encourages collaboration and adoption of the latest technologies. NVIDIA AI, through its official X account, retweeted Red Hat AI's announcement, signaling support for the initiative. The collaboration between Red Hat and NVIDIA in delivering DFlash checkpoints demonstrates synergy between AI infrastructure providers and model developers to improve inference efficiency. From a technical standpoint, DFlash uses a lightweight diffusion based draft model to generate blocks of tokens in parallel. The verifier model then validates all generated tokens in a single forward pass. This approach differs from traditional speculative decoding, which typically generates one token per step. With DFlash, the number of forward passes can be drastically reduced, especially for models with large parameter counts.

Validation on the NVIDIA B200 shows that the checkpoints perform optimally on NVIDIA's latest hardware. The B200 is a next generation GPU designed for AI and HPC workloads. With this validation, users can be confident that the checkpoints will deliver maximum performance on NVIDIA infrastructure. Red Hat AI also provided direct links to Hugging Face for downloading both checkpoints: Nemotron Ultra 550B and Nemotron Super 120B. This makes it easy for users to integrate the checkpoints into their pipelines. vLLM, as an inference framework that supports DFlash, provides a simple API to enable speculative decoding. Looking ahead, the development of speculative decoding like DFlash could become the standard for large model inference. As AI models grow larger, inference efficiency becomes key to widespread industry adoption.

Red Hat AI and NVIDIA continue to push the boundaries of inference capability through open innovation accessible to the community. For developers wanting to try DFlash, the steps are straightforward: download the checkpoints from Hugging Face, install vLLM with DFlash support, and run with the appropriate flags. Further documentation is available in the vLLM repository and on the Hugging Face pages for each checkpoint. With this release, Red Hat AI strengthens its position as a provider of open source AI solutions focused on performance and accessibility. Meanwhile, NVIDIA continues to expand the Nemotron model ecosystem with support from partners like Red Hat. This collaboration is expected to accelerate the adoption of speculative decoding among AI researchers and practitioners.