Crawlers
Every search engine, AI model and SEO tool that reads this site, across the website, the API and the image CDN. Updated every hour.
Scoped to ap-proxy.mangabaka.dev · every host
Requests
5.5k
Served
355 MB
Providers
17
Share of bots
0.1%
By what they are for
- AI Crawler 3.1k 238 MB
- SEO tooling 1.7k 74.8 MB
- AI search 350 24.8 MB
- Search engines 206 10.3 MB
- Archiver 57 5.1 MB
- Page Preview 27 112 KB
- Accessibility 12 805 KB
- Aggregator 3 435 KB
AI Crawler are 57% of automated traffic.
Which provider hits it
- Meta 3.0k 190 MB
- Semrush 681 31.5 MB
- Reflection AI 665 53.5 MB
- Anthropic 418 27.2 MB
- OpenAI 311 22.8 MB
- Apple 128 9.4 MB
- Amazon 117 10.1 MB
- Microsoft 55 4.2 MB
- Internet Archive 30 2.5 MB
- Google 30 2.3 MB
Meta is 55% of its requests.
What they take away
- webp 2.7k 201 MB
- jpeg 2.3k 152 MB
- html 478 1.8 MB
- empty 4 0 B
- ico 1 2.5 KB
- css 1 6.3 KB
Images are 91% of requests and 99% of bytes.
What we answered
- 200 OK 5.0k 353 MB
- 404 gone 295 1.1 MB
- 500 error 85 70.0 KB
- 403 refused 59 296 KB
- 502 bad gateway 39 271 KB
- 499 hung up 4 0 B
8.7% of bot requests are refused or errored.
Who crawls us
By requests
Meta
AI Crawler
+9169%
3.0k
190 MB 55%
Semrush
SEO tooling
−35.3%
681
31.5 MB 12%
Reflection AI
AI Crawler
−31.9%
665
53.5 MB 12%
Anthropic
AI Crawler
+51.4%
418
27.2 MB 7.6%
OpenAI
AI Crawler
−60.8%
311
22.8 MB 5.6%
Apple
AI search
−58.3%
128
9.4 MB 2.3%
Amazon
AI Crawler
−31.2%
117
10.1 MB 2.1%
Microsoft
Search engines
+19.6%
55
4.2 MB 1.0%
Requests per day
ap-proxy.mangabaka.devBytes per day
ap-proxy.mangabaka.devCounts come from Cloudflare's edge across both zones, refreshed every hour. Only bots Cloudflare has verified are named, so anything claiming to be Googlebot without the network to prove it is left out. Our own monitoring is excluded.