Google's DiffusionGemma proves you don't need to train

What happened

Building a new model from scratch wasn't necessary. The team started with the existing Gemma-4-26B-A4B and converted it into a diffusion model using less than ten percent of the original training token budget, according to the report. In the first of two steps, the model learns to reconstruct noisy text blocks from example data. A combined phase of reinforcement learning and sampler distillation follows, which Google calls SD·RL.

The subsequent training phase was relatively short, and the second step, SD·RL, prioritized speed over peak quality. The architecture, training data, and other settings were also carried over from the original Gemma 4 model, which aren't necessarily ideal for diffusion. The model occasionally gets stuck in repetition loops, producing individual words multiple times in a row. This is an artifact of the aggressively reduced compute steps.

Why it matters

Reinforcement learning typically boosts answer quality, while sampler distillation lets the model get by with fewer compute steps. Google merges both into a single process. According to the report, this combined approach raises quality on reasoning benchmarks by an average of ten points while nearly quadrupling the number of tokens per compute step. As a side effect, DiffusionGemma's answers run about 50 percent shorter, which further boosts speed.

Standard language models have to commit to the first digit of an answer before they've worked through the reasoning. In a math problem from the report, Gemma 4 starts its response with "-1," realizes during its derivation that "-25" is correct, and tacks on a correction afterward. DiffusionGemma develops the answer and reasoning in parallel, so it can fix mistakes before the output is finalized.

Sudoku solving works on the same principle, since every entry depends on entries that come later. After minimal fine-tuning, DiffusionGemma solves close to 85 percent of puzzles correctly, while the base model fails at the task entirely. Structured outputs like JSON or code repairs finish after just two to three refinement steps, according to the report, because the input already determines most tokens.

DiffusionGemma also keeps its original ability to generate text word by word, letting users switch between both modes depending on the task. Absolute performance falls short of the autoregressive base model. Google points to several reasons for this. DiffusionGemma wasn't trained as a diffusion model from the start but was retrofitted after the fact.

What to watch

On multimodal tasks, DiffusionGemma sometimes forgets to close its reasoning section properly, which artificially drags down benchmark scores. The speed advantage also holds mainly for single-user scenarios. Once about 32 concurrent requests hit the model, standard language models catch up on throughput.

Google explicitly calls DiffusionGemma an experimental model and says the release is meant to speed up research on text diffusion while giving the community a foundation for specialized, resource-efficient adaptations. The model is already being used by the startup Interfaze for multilingual speech recognition and in a research project on interactive radiology report generation. 0 license on Hugging Face. Its predecessor is Gemini Diffusion, which Google demoed in May 2025.