ELSEIF
Your brief EB
285 stories from 72 feeds 54 clusters Refreshed 11 minutes ago next pull 20:35

OBSERVABILITY Signal 281

How Mercado Libre rebuilt its observability platform on ClickHouse Cloud with 50x faster trace queries

Mercado Libre migrated its observability platform to ClickHouse Cloud, cutting trace query times from over five minutes to about four seconds (a 50x speedup) with up to 89% data compression while scaling ingest from 7 million to 400 million spans per minute.

WHY IT MATTERS

Engineers can now troubleshoot payment failures and user errors in minutes instead of days, thanks to end-to-end trace visibility and high-cardinality filtering on IDs like payment and user. The platform’s compression and scalable ingest keep observability costs aligned with business growth, making it feasible to retain full-fidelity data for critical services.

Written by elseif from the cluster below · every claim links back to a source

The three things worth knowing

01

The new design stores traces in ClickHouse Cloud, moving heavy transformations, filtering, and aggregation from client-side query time to server-side processing.

02

It is built on OpenTelemetry with 100% sampling for critical applications and business-flow attributes baked into the instrumentation.

03

Adoption required re-architecting the telemetry pipeline to ingest via ClickHouse Cloud and ongoing capacity planning to sustain performance as trace volume grows.

THE READ

What elseif makes of it.

ORIGINAL ANALYSIS

Mercado Libre’s original observability stack relied on an OpenTelemetry collector feeding a storage layer where query-time transformations, filtering, and aggregation were performed on the client, creating blockers that limited throughput to about 70 million spans per minute. By moving to ClickHouse Cloud, the team shifted those operations to the database engine, which eliminated the client-side bottleneck and allowed query latency to drop from minutes to seconds. The redesign also introduced columnar storage that provides the up to 89% data compression reported.

The migration involved redesigning the ingestion path to write directly into ClickHouse Cloud, defining schemas that support high-cardinality fields such as payment and user IDs. Retention policies were configured to balance cost with the need for full-fidelity traces on critical services. This work required the observability team to learn ClickHouse-specific tuning and to allocate cloud resources that match the observed ingest rate.

Performance gains are tied to the provisioned capacity of ClickHouse Cloud; if the incoming span rate exceeds the allocated compute or storage bandwidth, query times can increase and compression efficiency may drop. Extremely ad-hoc or low-volume debugging scenarios may not benefit as much from the analytics-optimized layout, and the platform may be over-provisioned for such use cases. The solution assumes that the trace volume will continue to grow within the planned scaling envelope, requiring periodic capacity reviews.

Despite these limits, the new O11y events platform delivers 100% sampling for critical paths, end-to-end visibility of business flows, and the ability to filter on any identifier without pre-aggregation. These capabilities turn incident response from a service-centric view to a precise root-cause explanation, such as linking a failed payment to a specific service timeout. Reducing mean time to resolution lets engineers correlate telemetry with business outcomes such as failed payments or checkout errors. The platform also supports trend analysis, anomaly detection, and performance monitoring as expected from an observability system at Mercado Libre’s scale.

Written by elseif from the cluster below · checked for specifics the sources never contained

THE CLUSTER

Same story, 1 feed.

ORDERED BY FIRST SEEN
ClickHouse How Mercado Libre rebuilt its observability platform on ClickHouse Cloud with 50x faster trace queries Open ↗