Show HN: Maple-Preview – ternary 20B MoE running at 120 tok/s on a iPhone

The headline announces a technical milestone: a 20-billion-parameter mixture-of-experts model running at 120 tokens per second on an iPhone. This is notable because it demonstrates that large-scale language models can operate efficiently on consumer mobile hardware, not just high-end servers or cloud instances. The use of ternary quantization, reducing model weights to three possible values, likely plays a key role in achieving this performance by cutting memory bandwidth and compute overhead.

For engineers building AI-powered applications, this changes the feasibility of deploying advanced models in offline or low-latency environments. Previously, running models of this scale on a phone would have been impractical due to power, thermal, and memory constraints. The demo suggests these barriers are being overcome, but it does not specify the model’s accuracy or the hardware generation used, leaving questions about real-world usability.

The trade-offs here are critical. Ternary quantization reduces precision, which can degrade model quality, especially for tasks requiring nuanced outputs. The 120 tokens per second figure is impressive, but it may not hold for all iPhone models or under sustained load. Engineers will need to evaluate whether the speed gain justifies potential accuracy loss and whether the approach generalizes beyond this specific demo.