PERFORMANCE Signal 168
Systems Engineering Playbook: Optimizing Qwen 3.5-397B MoE on Ironwood (TPU7x)
↗
Decomposing large models into reusable, hardware-aware building blocks allows teams to optimize new architectures rapidly rather than treating them as monolithic systems. This approach yielded up to a 4.7x speedup for prefill-heavy workloads and integrates directly into open-source serving frameworks like vLLM and SGLang, reducing friction for production deployments.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The optimization strategy treats models as collections of independent building blocks, enabling near-zero engineering friction when porting pre-optimized modules to new architectures.
Targeting Qwen 3.5's novel components, such as Gated DeltaNet linear attention and Attention Data Parallelism, resulted in a 3.1x speedup for decode-heavy workloads and a 4.7x speedup for prefill-heavy workloads.
The Qwen 3.5 architecture activates only 17 billion of its 397 billion total parameters per token by using a highly sparse routing scheme and a hybrid layout of Gated DeltaNet and Grouped Query Attention layers.
THE CLUSTER