Crawlers
Every search engine, AI model and SEO tool that reads this site, across the website, the API and the image CDN. Updated every hour.
2026-10-11 to 2026-10-17
Requests
2.5M
Served
180 GB
Providers
69
Hosts
13
By what they are for
- AI Crawler 1.1M 66.9 GB
- Search engines 758k 42.8 GB
- AI search 407k 45.6 GB
- Monitoring & Analytics 108k 7.6 GB
- SEO tooling 94.4k 11.2 GB
- Page Preview 60.6k 1.2 GB
- Archiver 23.0k 1.7 GB
- Security 9.5k 948 MB
- AI assistant 4.1k 2.1 GB
- Accessibility 922 105 MB
AI Crawler are 42% of automated traffic.
Which host they hit
- mangabaka.org 1.6M 71.0 GB
- cdn.mangabaka.dev 514k 66.6 GB
- api.mangabaka.org 400k 3.8 GB
- images.mangabaka.dev 47.2k 38.3 GB
- ap-proxy.mangabaka.dev 4.3k 265 MB
- mangabaka.dev 2.7k 14.9 MB
- analytics.mangabaka.dev 531 1.5 MB
- uploads.mangabaka.org 187 44.3 MB
- api.mangabaka.dev 90 111 KB
- www.mangabaka.org 86 92.6 KB
38% of bot requests never touch the website.
What they take away
- html 1.4M 69.2 GB
- json 291k 1.1 GB
- jpeg 266k 45.1 GB
- webp 189k 47.4 GB
- empty 173k 262 MB
- atom 110k 257 MB
- avif 66.5k 1.3 GB
- png 40.4k 11.5 GB
- js 32.6k 1.0 GB
- css 10.6k 97.3 MB
Images are 22% of requests and 58% of bytes.
What we answered
- 200 OK 2.1M 179 GB
- 403 refused 249k 935 MB
- 301 moved 158k 179 MB
- 404 gone 13.8k 209 MB
- 302 found 11.0k 12.6 MB
- 308 slash 2.9k 2.4 MB
- 304 unchanged 2.2k 2.2 MB
- 206 partial 1.2k 9.3 MB
- 502 bad gateway 543 2.7 MB
- 499 hung up 393 0 B
10% of bot requests are refused or errored.
Who crawls us
By requests
Anthropic
AI Crawler
+1197%
491k
26.5 GB 19%
Meta
AI Crawler
−62.6%
402k
9.9 GB 16%
Apple
AI search
+0.3%
298k
43.8 GB 12%
ExaSearchBot
Search engines
+115%
252k
11.9 GB 9.9%
Microsoft
Search engines
−10.6%
239k
20.7 GB 9.4%
OpenAI
AI Crawler
−61.0%
208k
21.4 GB 8.2%
Parallel
AI search
−67.6%
108k
1.8 GB 4.2%
DataDog
Monitoring & Analytics
−0.9%
108k
7.6 GB 4.2%
Requests per day
all verified bots, every hostBytes per day
all verified bots, every hostCounts come from Cloudflare's edge across both zones, refreshed every hour. Only bots Cloudflare has verified are named, so anything claiming to be Googlebot without the network to prove it is left out. Our own monitoring is excluded.