llms.txt proposed as AI-readable site guide, but no major AI platform confirms usage

llms.txt is proposed as a simple Markdown file placed at the root of a website that lists the most important pages for AI systems to prioritize. The file is not an official standard and no major AI platform has publicly confirmed that it reads or acts on its contents. Its purpose is to give agents a machine-readable overview of a site’s authoritative content. Unlike robots.txt, it does not restrict access but merely suggests what to read first.

Adoption data show limited uptake: an SE Ranking study of 300,000 domains found a 10.13% prevalence, and among the fifty domains most often cited by AI, only one had the file. Monitoring of over 500 million AI bot events by Limy.ai recorded only a few hundred direct requests to /llms.txt over a 90-day window. The overwhelming majority of crawler traffic still targets HTML pages instead. These figures indicate that the file is largely ignored by current AI agents.

Despite the low usage, creating an llms.txt file requires only about twenty minutes of effort. The process forces teams to write concise descriptions of their top pages, which can surface duplication, orphaned content, or pages that lack substantive information. Several teams report gaining more value from this audit than from the file itself. The file also represents a cheap optional position should agentic routing ever standardize on such a machine-readable surface.

However, the file stops working as a reliable signal because major providers such as Google have explicitly stated they do not support llms.txt and have likened it to the discredited keywords meta tag. A significant share of existing files are junk defaults generated by plugins rather than curated lists, further diluting any potential benefit. Engineers should prioritize improving crawl access, content quality, and answer structure before investing time in llms.txt. Relying on the file for AI citation or traffic gains is not supported by the available evidence.