Crawlers
Every search engine, AI model and SEO tool that reads this site, across the website, the API and the image CDN. Updated every hour.
Scoped to ap-proxy.mangabaka.dev · every host
The period being compared against reaches back further than our records, which start 2026-08-28. Changes marked * read larger than they were.
Requests
19.8k
Served
179 MB
Providers
14
Share of bots
0.3%
By what they are for
- AI Crawler 18.2k 113 MB
- SEO tooling 1.3k 50.3 MB
- Search engines 275 11.7 MB
- Security 26 1.6 MB
- Archiver 11 901 KB
- Aggregator 7 931 KB
- AI search 5 164 KB
- Page Preview 5 20.5 KB
- Accessibility 3 302 KB
AI Crawler are 92% of automated traffic.
Which provider hits it
- OpenAI 16.6k 15.8 MB
- Semrush 1.3k 50.0 MB
- Anthropic 1.0k 37.7 MB
- Amazon 761 60.0 MB
- Microsoft 119 8.9 MB
- Yandex 44 2.0 MB
- Unknown 26 1.6 MB
- Internet Archive 11 901 KB
- Pinterest 7 931 KB
- Google 7 357 KB
OpenAI is 83% of its requests.
What they take away
- txt 16.9k 15.1 MB
- webp 1.3k 86.3 MB
- jpeg 1.3k 76.3 MB
- html 308 1.3 MB
- ico 1 2.5 KB
- css 1 6.3 KB
Images are 13% of requests and 91% of bytes.
What we answered
- 403 refused 16.9k 15.1 MB
- 200 OK 2.6k 163 MB
- 404 gone 288 1.2 MB
- 500 error 17 13.9 KB
- 301 moved 1 882 B
87% of bot requests are refused or errored.
Who crawls us
by requestsOpenAI
AI Crawler
+3177%*
16.6k
15.8 MB 83%
Semrush
SEO tooling
+63.2%*
1.3k
50.0 MB 6.5%
Anthropic
AI Crawler
+828%*
1.0k
37.7 MB 5.0%
Amazon
AI Crawler
+1564%*
761
60.0 MB 3.8%
Microsoft
Search engines
+35.7%*
119
8.9 MB 0.6%
Yandex
Search engines
44
2.0 MB 0.2%
Unknown
Security
26
1.6 MB 0.1%
Internet Archive
Archiver
+1000%*
11
901 KB 0.1%
Requests per day
ap-proxy.mangabaka.devBytes per day
ap-proxy.mangabaka.devCounts come from Cloudflare's edge across both zones, refreshed every hour. Only bots Cloudflare has verified are named, so anything claiming to be Googlebot without the network to prove it is left out. Our own monitoring is excluded.