How iFood built its agentic security platform on ClickHouse Cloud

iFood’s decision to rebuild its security platform on ClickHouse Cloud was driven by scalability and cost constraints with its previous Databricks-based system. The security team needed to ingest and retain large volumes of logs while maintaining fast query performance for incident response. Databricks, while effective for BI and data science, became prohibitively expensive as log volumes grew past 130 TB, forcing query timeouts and budget caps that limited detection capabilities. ClickHouse Cloud addressed these issues by offering a lower-cost, high-performance alternative that could handle the same workload without trade-offs.

The performance gains were substantial: queries ran 9-16x faster at 40-50% of the cost, and data freshness improved from hourly batches to near real-time updates. This shift unlocked agentic threat hunting, where automated sub-agents now query 30+ TB of data in parallel, reducing a week’s worth of analyst work to two hours. The migration also simplified the architecture, using ClickPipes for log ingestion from AWS S3, which provided reliable, low-latency data delivery without requiring Kafka for most use cases.

For engineers, the key takeaway is the importance of aligning database choice with specific workload requirements. Databricks remained the foundation for iFood’s Lakehouse, but ClickHouse Cloud proved better suited for security operations due to its speed, cost efficiency, and ability to scale log retention. The trade-off was not in functionality but in specialization: ClickHouse Cloud’s design for analytical queries at scale made it a better fit for iFood’s security needs, while Databricks continued to serve broader data science and BI use cases.