PERFORMANCE Signal 251 2 feeds carried it
Apple M3 Neural Engine DRAM throughput reportedly throttles at 1 MiB weight multiples restoring 27 GB/s gain
↗
Illustration only Photo by Monisha Selvakumar on Unsplash
An RTL erratum in the Apple M3 Neural Engine reduces DRAM weight streaming throughput to 17 to 19 GB/s at 1 MiB-aligned weight sizes affecting 7 of 15 ANEMLL models
Engineers running small-batch inference on M3 Macs can recover 2 to 3× token throughput by avoiding 1 MiB-aligned weight tensors. The fix is a one-line kernel change but requires retraining or padding models that hit the erratum. No silicon revision is available yet so the workaround remains necessary for affected models.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Throughput drops from 45 to 60 GB/s to 17 to 19 GB/s when total weight size is an integer multiple of 1 MiB
Padding or shrinking weight tensors by 256 elements restores full bandwidth and doubles token throughput
The erratum is present in 7 of the 15 ANEMLL models shipped with M3 Macs
THE READ
What the cluster adds up to.
The Apple M3 Neural Engine contains an RTL performance erratum that throttles DRAM weight streaming throughput to 17 to 19 GB/s whenever the total weight size is an exact multiple of 1 MiB. This affects 7 of the 15 ANEMLL models currently deployed. The erratum is not a correctness bug, transfers complete correctly, but the DMA engine is forced into a credit-starved issue regime at these boundaries, costing 28 to 43 GB/s of bandwidth.
Engineers can work around the issue by adjusting weight tensor dimensions. Shrinking or padding the dimension by 256 elements (≈1 KiB) restores nominal throughput of 45 to 60 GB/s. For example, Llama 3.2 1B token throughput increased from 10.0 to 24.3 tokens/s, and Qwen3-8B from 1.36 to 2.97 tokens/s, simply by avoiding the problematic alignment. The fix requires no hardware changes but may necessitate retraining or model modification.
The erratum manifests as a dominant harmonic in throughput measurements, with a sharp dip at multiples of 2048 elements. This suggests a spatial correlation issue in the DRAM controller, where parallel requests from the 16 Neural Engine cores alias onto the same DRAM bank at power-of-two strides. The workaround effectively redistributes requests across banks, restoring parallelism and full bandwidth.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER