TECH Signal 313
Only 8.9% of sites block AI crawlers, but 94.8% are never cited in AI answers
↗
For site operators, the critical distinction is between training crawlers and retrieval agents: blocking the former costs nothing in current citations, but blocking the latter makes a site uncitable. The practical bottleneck is that only 19.3% of audited sites publish LocalBusiness schema and nearly half lack any structured data, leaving AI assistants with nothing but prose to interpret when deciding whom to name.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Only 1.9% of 5,978 assistant answers named the queried business, with Gemini citing the business in 2.9% of answers compared to 1.6–1.7% for the other three assistants tested.
Sites overwhelmingly block training crawlers like GPTBot (38 sites) and ClaudeBot (36 sites) while rarely blocking retrieval agents like OAI-SearchBot (4 sites) and Perplexity-User (3 sites), though confusing the two categories risks making a site invisible to live answers.
Just 19.3% of sites publish LocalBusiness schema and 54.6% publish any schema.org structured data, meaning most local businesses provide no machine-readable identity or location information for AI systems to extract.
THE CLUSTER