Free Tool · No Sign-Up
Free AI Crawler Access Checker
Enter your domain. We read your robots.txt and tell you exactly which AI crawlers — GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and more — are blocked from indexing your pages.
This checks 1 thing. HellSEO's full audit scores 24 checks across SEO, AEO, and GEO — including backlink authority and score history this page doesn't cover.
See Full ScoreHow It Works
We fetch your site's robots.txt and parse every User-agent / Disallow group in it — the same rules search engines and AI crawlers read before touching your site.
For each known AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others), we check whether that specific user-agent — or a wildcard * covering everyone — is blocked with a full Disallow: /.
This only reads your robots.txt; it doesn't crawl your pages or store anything. Free, and it runs in a couple of seconds.
Who Uses This
SaaS & product sites
Confirm ChatGPT and Perplexity can actually read your docs and pricing pages before assuming AI traffic will find you.
Publishers & blogs
Decide deliberately whether to allow AI training crawlers (CCBot, Bytespider) instead of inheriting a default block from your CMS.
Agencies
Add this to a client onboarding checklist — a blocked crawler is a one-line fix that's easy to miss.
Frequently Asked Questions
What happens if I block an AI crawler?
That bot can no longer read your pages, so your content can't appear in that engine's answers — even if you rank #1 on Google. Many CMS default robots.txt files block these bots without you knowing.
Does blocking GPTBot also block ChatGPT search?
No. OpenAI uses separate user-agents — GPTBot for training data, OAI-SearchBot for live ChatGPT search results. You can block one and allow the other.
Will this affect my Google ranking?
No. Google's classic crawler (Googlebot) is separate from Google-Extended, which only feeds Gemini and AI Overviews. Blocking Google-Extended doesn't affect normal search ranking.
How do I unblock a bot?
Remove or edit the matching User-agent / Disallow block in your robots.txt file, then re-check here to confirm.
Want the full picture?
This tool checks one thing. A full HellSEO audit scores your site across SEO, AEO, and GEO — and gives you ready-to-paste fixes.
Run a Full Audit