TECH Signal 493
DiffusionGemma introduced as open-weight model using discrete diffusion to generate 256-token blocks in parallel
↗
Illustration only Photo by Brad Helmink on Unsplash
The DiffusionGemma report describes an open-weight language model that replaces token-by-token decoding with parallel refinement of 256-token blocks, achieving roughly 1,500 output tokens per second on a single H100 GPU.
Engineers can obtain much higher generation throughput without sacrificing the capabilities of the underlying Gemma 4 model, which is valuable for real-time applications. The approach also demonstrates a compute-efficient training pipeline, using under 10 % of the original model’s token budget, suggesting lower cost for future model development.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
DiffusionGemma refines blocks of 256 tokens per forward pass, delivering about 20 tokens per pass and ~1,500 tokens per second on an H100.
The model is fine-tuned from the Gemma 4 mixture-of-experts model (3.8 B active, 25.2 B total parameters) via a two-stage pipeline that consumes less than 10 % of the original token budget.
It retains Gemma’s support for thinking mode, multimodal inputs, long contexts, and can still generate autoregressively with only minor performance loss.
THE CLUSTER