Google's DiffusionGemma generates text in parallel blocks, claiming 4x GPU speedup

Google has released DiffusionGemma, an experimental 26B Mixture-of-Experts model under Apache 2.0 that abandons the token-by-token decoding of conventional LLMs in favor of text diffusion. Borrowing the iterative-refinement approach of image generators, it drafts 256-token blocks simultaneously and polishes them over multiple passes, hitting 1,000+ tokens per second on an H100 and 700+ on an RTX 5090. With only 3.8B parameters active at inference, the quantized model fits in 18GB of VRAM, putting it within reach of high-end consumer cards.

The design targets a specific weakness of autoregressive models: single-user local inference, where sequential decoding leaves GPUs mostly idle waiting on memory bandwidth. By shifting the bottleneck to compute, diffusion decoding saturates local hardware — but Google is explicit that the advantage evaporates in high-throughput cloud serving, where batched autoregressive models already use compute efficiently and parallel decoding can actually raise costs. Output quality also trails standard Gemma 4, so Google positions this as a research and prototyping tool rather than a production model.

The more interesting angle may be bi-directional attention: because every token in a block attends to every other, the model handles tasks where later tokens constrain earlier ones — code infilling, in-line editing, structured outputs, even Sudoku in a fine-tuning demo from Unsloth. Launch-day tooling includes vLLM, MLX, and Hugging Face Transformers support, NVIDIA NVFP4 optimization for Hopper and Blackwell, and a JAX fine-tuning toolbox, signaling Google wants the ecosystem to probe whether text diffusion can graduate from research curiosity to practical architecture.