AI Signal 502
LiquidAI releases QAD-trained Q4_0 GGUF checkpoints for LFM2.5 models with near-BF16 accuracy
↗
LiquidAI published 4-bit QAD-trained GGUF checkpoints for four LFM2.5 model sizes that recover about 97 % of the BF16 baseline accuracy while keeping Q4_0 memory and speed.
Engineers can now run LFM2.5 models on edge devices with the low memory footprint of 4-bit quantization but without the typical quality loss, simplifying deployment on constrained hardware. The checkpoints deliver higher decode throughput than comparable post-training quantizations, reducing latency for real-time applications.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Quantization-Aware Distillation recovers roughly 97 % of the BF16 accuracy lost to quantization for all four LFM2.5 checkpoints.
The Q4_0 checkpoints retain the same memory footprint and decode speed as native Q4_0 GGUFs.
On edge hardware the QAD checkpoints achieve 3-33 % higher throughput while matching the quality of higher-bit quantizations.
THE READ
What the cluster adds up to.
LiquidAI introduced a new set of QAD-trained Q4_0 GGUF checkpoints for the LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B models. These checkpoints are produced by distilling a high-precision teacher into a quantized student, replacing earlier post-training quantization (PTQ) versions. The release is announced as a way to run the models at Q4_0 memory and speed without the usual quality drop.
Benchmarking across reasoning, instruction-following, tool use, and agentic tasks shows the QAD checkpoints retain 96.5-97.4 % of their BF16 baseline performance. The authors compare the QAD checkpoints to PTQ GGUFs and report that the QAD versions substantially improve the Q4_0 checkpoint across the suite. The BF16 GGUF serves as the in-format ceiling for these comparisons.
Real-world decode throughput was measured on a MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. The 230M and 350M QAD checkpoints match Q5_K_M quality while delivering 4-33 % higher throughput; the 1.2B and 2.6B checkpoints match Q4_K_M quality with 3-14 % higher throughput. They also match Unsloth's UD-Q4_K_XL checkpoints where applicable, confirming competitive performance on both GPU and Arm CPU targets.
To use the new checkpoints, developers must employ llama.cpp or any runtime that supports GGUF Q4_0 artifacts, as shown in the provided command line example. No additional memory is required beyond the native Q4_0 footprint, but the runtime must understand the GGUF format. Systems lacking GGUF support will not benefit from these checkpoints without adding compatible software.
For edge deployment, the QAD checkpoints enable larger LFM2.5 models (up to 2.6 B parameters) to run on devices with limited RAM while preserving most of the original model quality. The remaining ~3 % accuracy gap may still affect highly sensitive tasks, so engineers should evaluate whether the trade-off meets their application requirements. Nonetheless, the release lowers the hardware barrier for sophisticated language models on consumer-grade devices.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER