PERFORMANCE Signal 338
Running Kimi K3 on MI355X at Better Performance per Dollar Than B300
↗
For teams deploying frontier-scale models, the MI355X's 288GB VRAM enables single-node deployments that would require multi-node setups on B200s, yielding a practical cost advantage at 48 tok/s/$ versus B300's 33 tok/s/$. The tradeoff is significantly slower prefill performance and lingering ROCm software gaps that demand engineering effort, such as a missing top-k renorm function that crashed the speculative decode scheduler.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The MI355X achieves 48 tok/s/$ peak aggregate throughput versus B300's 33 tok/s/$ and B200's 7 tok/s/$, driven by pricing of $2.50/GPU-hr versus $6.00 and $4.25 respectively, even though B300 wins on absolute throughput by roughly 1.65×.
A missing top-k renorm definition in ROCm's sglang sampling branch crashed the scheduler during speculative decode; the fix was a simple PyTorch function, not a custom kernel, highlighting that AMD software gaps are often mundane missing definitions rather than fundamental hardware limitations.
The MI355X takes roughly 51 seconds for a 172k-token cold prefill versus the B300's 23 seconds, meaning workloads with large context prefills will suffer worse time-to-first-token on AMD despite competitive decode speeds.
THE CLUSTER