Bot Shield

FREE

Stop AI crawlers from burning your hosting budget and training on your content. Analyze, block, and rate-limit GPTBot, ClaudeBot, and 12 other AI scrapers.

GPTBot hit one developer's site with 164,000 requests in a single day — forcing a $20/mo Vercel Pro upgrade.

Your daily AI bot requests:≈ $23.13/mo on Vercel

AI bots tracked

Bots that respect robots.txt

11/14

Bots that ignore it

Analyze your robots.txt

Paste your current robots.txt to see which AI bots you're blocking, which are getting through, and the estimated bandwidth impact.

100% client-side. Nothing leaves your browser.

AI Bot Reference

All 14 tracked AI crawlers, their bandwidth footprint, and whether robots.txt actually stops them.

Bot	Company	Bandwidth	Risk	Respects robots.txt
GPTBot GPTBot	OpenAI	100K–500K req/day	critical	Yes
ChatGPT-User ChatGPT-User	OpenAI	1K–10K req/day	medium	Yes
ClaudeBot ClaudeBot	Anthropic	10K–100K req/day	high	Yes
anthropic-ai anthropic-ai	Anthropic	10K–50K req/day	high	Yes
Google-Extended Google-Extended	Google	5K–50K req/day	high	Yes
PerplexityBot PerplexityBot	Perplexity AI	5K–30K req/day	high	Yes
Amazonbot Amazonbot	Amazon / AWS	10K–50K req/day	high	Yes
Bytespider Bytespider	ByteDance (TikTok)	50K–200K req/day	critical	No — use middleware
cohere-ai cohere-ai	Cohere	1K–10K req/day	medium	Yes
Meta-ExternalAgent Meta-ExternalAgent	Meta	10K–50K req/day	high	Yes
Diffbot Diffbot	Diffbot	1K–20K req/day	medium	No — use middleware
CCBot CCBot	Common Crawl	10K–100K req/day	critical	Yes
FacebookBot facebookexternalhit	Meta	1K–10K req/day	low	No — use middleware
Applebot-Extended Applebot-Extended	Apple	1K–10K req/day	medium	Yes