INFRA Signal 203
Your site, your rules: new AI traffic options for all customers
↗
Site operators no longer have to choose between discoverability and content protection—they can allow search indexing while blocking training crawlers. This gives engineers finer-grained access control over automated traffic hitting their infrastructure and monetized content.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The new taxonomy classifies bots by behavior—Search, Agent, or Training—rather than by whether they qualify as AI, enabling differentiated access policies per use case.
Cloudflare is encouraging bot operators that serve multiple purposes to split their automation into separate crawlers so site owners can understand and manage each purpose independently.
All customers now get these granular controls plus the ability to protect ad-monetized pages, moving beyond the previous one-size-fits-all block option.
THE CLUSTER