Anthropic engineer reports LLMs excel at log observation but fail root-cause analysis in incident response

Anthropic’s reliability engineer Alex Palcuie provides a field report on using LLMs for incident response, focusing on real-world outcomes rather than theoretical capabilities. The presentation identifies a clear division of labor: LLMs act as a ‘superhuman’ observer for logs and traces, processing vast amounts of data faster than human engineers. This advantage accelerates initial triage but does not extend to root-cause analysis, where the model’s inability to distinguish correlation from causation becomes a critical limitation. The gap persists even for Anthropic, which has privileged access to its own models and training data.

The practical cost of adopting LLMs in incident response is the need for continuous human oversight. Palcuie’s team still carries pagers and hires reliability engineers, demonstrating that AI tools do not yet automate away the need for human judgment. The risk lies in over-reliance on LLMs for diagnosis, which could lead to misdiagnosis or delayed resolution of complex incidents. Engineering leaders must design workflows that preserve human expertise while leveraging AI for tasks where it excels, such as log summarization or anomaly detection.

The presentation highlights where LLMs stop working effectively in incident response. Root-cause analysis remains a human task because it requires contextual understanding and hypothesis testing, areas where LLMs currently lack robustness. Additionally, LLMs may struggle with novel or edge-case incidents that fall outside their training data, requiring engineers to step in. The reliability team’s experience suggests that AI tools are best suited for augmenting, not replacing, human responders in high-stakes environments.

The broader implication for engineers is the need to critically evaluate AI tooling in production systems. While LLMs can improve response times for routine tasks, their limitations in diagnosing complex issues demand structured integration. Palcuie’s skepticism toward ‘AI SRE’ startups underscores the gap between marketing claims and real-world performance. Teams adopting these tools must validate their effectiveness in their own environments, rather than assuming transferability from benchmarks or curated datasets.