Home / Reports / AI training signal

AI training signal tracker

Of 8,813 websites the registry has checked for it, 17 use the robots.txt AI training preference to opt out of AI training, in whole or in part.

What robots.txt says	Sites	Share
Opted out of AI training	16	0.2%
Opted out for some paths	1	0.0%
Allows AI training	0	0.0%
No preference stated	8,796	99.8%

Counted from the registry's own plain HTTP checks, checked since 7 November 2026. Every tracked site is re-checked about every 14 days, so this fills in as the token spreads. Download every site: ai-training-signal.csv.

What the signal is

A robots.txt group headed User-Agent: __usage_ai-training states a preference about AI training, not about crawling: Disallow: / means "do not train on this site", path rules can mix allow and deny, and a file with no such group states no preference. It comes from an individual Internet-Draft (draft-kelley-aipref-training-preference-signal), not yet an IETF standard; major AI providers have said they will honour it from early 2027.

Site owners can also state AI training, search and browsing permissions directly by claiming their site's entry; those are shown as the owner's own word beside what robots.txt says. Claim your site.

Sites that opt out

Use this from an agent

Every lookup carries it as robots_ai_training (deny, partial, allow, absent): /v1/origins/{host}.