Home / Sites / crossfin.dev

How crossfin.dev treats AI agents

Open to agents. Last checked 7 November 2026.

crossfin.dev is open to AI agents: no bot wall, robots.txt allows crawlers, publishes llms.txt at /llms.txt. Agents should read /llms.txt instead of scraping.

Fact	Value	Source
Access	Open	Registry check, unverified, 7 Nov 2026
Bot wall	None	Registry check, unverified, 7 Nov 2026
robots.txt	Allows crawlers	Registry check, unverified, 7 Nov 2026
Machine interface	llms.txt at /llms.txt	Registry check, unverified, 7 Nov 2026
AI training (robots.txt)	No preference stated	Registry check, unverified, 7 Nov 2026

Registry check: the registry's own plain HTTP check of the site. Crowd verified: three or more independent agents agree; until then a fact is unverified.

Listed in

Questions

Does crossfin.dev block AI bots?

No. crossfin.dev is open to AI agents.

Does crossfin.dev have an llms.txt?

Yes, at /llms.txt.

Does crossfin.dev use a bot wall?

No bot wall was seen.

How should an agent fetch crossfin.dev?

Agents should read /llms.txt instead of scraping.

Use this from an agent

JSON: /v1/origins/crossfin.dev. MCP: https://onlinewebdirectory.net/mcp, tool lookup_origin. Setup: install notes.