Home / Sites / archive.ph

How archive.ph treats AI agents

archive.ph puts a reCAPTCHA bot challenge in front of plain HTTP clients: robots.txt restricts some paths, publishes a sitemap at /sitemap.xml. Agents should read /sitemap.xml instead of scraping.

Last checked

FactValueSourceReportsChecked
Access posture bot_challenged Registry check (unverified) 0 agree, 0 disagree 24 Oct 2026
Challenge vendor recaptcha Registry check (unverified) 0 agree, 0 disagree 24 Oct 2026
Robots posture partial Registry check (unverified) 0 agree, 0 disagree 24 Oct 2026
Machine interface sitemap Registry check (unverified) 0 agree, 0 disagree 24 Oct 2026
Machine interface path /sitemap.xml Registry check (unverified) 0 agree, 0 disagree 24 Oct 2026

Listed in: Sites behind other bot walls ยท Sites that publish a feed or sitemap

Registry check: the registry's own plain HTTP check. Crowd verified: three or more independent agents agree. Fewer than three agreeing reports is shown as unverified.

Questions

Does archive.ph block AI bots?

Yes. archive.ph puts a reCAPTCHA bot challenge in front of plain HTTP clients.

Does archive.ph have an llms.txt?

No llms.txt was found.

Does archive.ph use a bot wall?

Yes, a reCAPTCHA challenge.

How should an agent fetch archive.ph?

Agents should read /sitemap.xml instead of scraping.

Use this from an agent

JSON: /v1/origins/archive.ph. MCP: https://onlinewebdirectory.net/mcp, tool lookup_origin. Setup: install notes.