AI robots.txt rules generator
Choose whether AI assistants can read your site to answer questions, and whether AI companies can use it for training, then copy the rules into robots.txt.
Your AI robots.txt Rules
Add these lines to your robots.txt, below any existing rules.
# AI search and assistants: allowed User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / # AI training: blocked User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: meta-externalagent Disallow: /
Questions
About The AI robots.txt rules
New to SEO? Read SEO Vs AEO Vs GEO: What They Mean And How To Do Each.
Does blocking Google-Extended remove me from Google?
No. Google-Extended only controls use in Gemini training; Google Search and AI Overviews use Googlebot.
AI search
All 22 Tools →Related Tools
Free · Up To 50 Pages
Check Your Whole Site, Free
Paste your homepage and CiteZap checks every page and writes the keywords, titles, descriptions, schema and fixes.