AI Signal 98
China Merchants Bank unifies AI training and inference on Kubernetes, lifting accelerator utilization from 35% to over 60%
↗
China Merchants Bank built a unified Kubernetes control plane combining Kueue, KEDA, Prometheus, HAMi, and Fluid to share nearly 10,000 heterogeneous accelerator cards across training, fine-tuning, and online inference, raising average utilization from 35% to over 60% and cutting inference cost per million tokens by more than 60%.
This is a production-scale demonstration that composable, vendor-neutral CNCF projects can decouple training and inference runtimes while sharing the same hardware pool, addressing the competing demands of stable training capacity and elastic inference scaling. The in-house Twinkle framework's multi-tenant LoRA sharing shows a concrete approach to reducing accelerator waste in fine-tuning. Only one feed carried this story, so the claims rest on a single source.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Bringing 99% of its accelerator resources under the unified framework lifted average utilization from 35% to over 60% across nearly 10,000 cards and cut inference cost per million tokens by more than 60%.
The in-house Twinkle training framework lets five LoRA tenants share one base-model instance by default, cutting accelerator resource usage for that setup by 80% while increasing training density fivefold.
Kueue manages training admission, queues, and quotas so jobs don't reserve capacity before they can use it, while KEDA and Prometheus scale online inference from live demand signals.
THE CLUSTER