Implement robots.txt
SkillCloud & infraSub-skill de agent-ready-cloudflare: Implement robots.txt
Available today. Use it from your connected AI after setup.
No other account needed.
Connect ahel once, and every AI you use reads what you have installed.
Then ask your AI: use the Implement robots.txt skill
What this skill tells your AI
The instructions your AI receives, as published by fabricioctelles/skills in skills/agent-ready-cloudflare/robots-txt/SKILL.md and read by ahel’s review.
Publish a valid robots.txt at your site root per RFC 9309.
Requirements
- Serve
/robots.txtastext/plainwith HTTP 200 - Include
User-agentdirectives withAllow/Disallowrules - Reference your sitemap if one exists:
Sitemap: https://example.com/sitemap.xml
Cloudflare
AI Crawl Control can manage your robots.txt from the dashboard, including AI-specific bot rules.
Validate
POST https://isitagentready.com/api/scan
Content-Type: application/json
{"url": "https://YOUR-SITE.com"}
Check that checks.discoverability.robotsTxt.status is "pass".
Signals
- GitHub stars
- 77
- Forks
- 7
- Last commit
- Sep 2026
Advanced
- Catalog kind
- skill
- Gateway key
agent-ready-robots-txt- Source
- github.com/fabricioctelles/skills