Not sure where to start? Take a guided tour.
AI Site Grader
AI Query Scanner
llms.txt GeneratorAI crawler traffic share by bot
GPTBot dominates the crawler landscape, but the behavior of smaller bots still matters because they enter and read sites differently.
Observed crawler traffic share across major AI bot families.
The key numbers.
- GPTBot share
- GPTBot is the dominant crawler by observed visits.
- 57.2%
- OAI-SearchBot share
- SearchBot is the second-largest crawler in the study.
- 15.1%
- ClaudeBot share
- ClaudeBot is smaller in raw volume but more diverse in behavior.
- 3.8%
- Crawler visits
- Observed AI crawler visits in the study window.
- 575,788
The ranked view.
- 157.2%
GPTBot
The largest crawler family in the study by visit share.
- 215.1%
OAI-SearchBot
The second-largest crawler family and a distinct search-oriented pattern.
- 39.2%
Bytespider
A meaningful third crawler with a very different profile.
- 47.9%
Meta-ExternalFetcher
Still material in raw traffic, but behavior is narrower.
- 53.8%
ClaudeBot
Smaller by volume, more diverse in where it lands.
What changed
OpenAI bot families dominate raw traffic
GPTBot plus OAI-SearchBot together account for the large majority of observed visits.
Volume is not the whole story
ClaudeBot matters because its homepage and page-type mix differs meaningfully from GPTBot.
| Measure | Result |
|---|---|
| GPTBot share GPTBot is the dominant crawler by observed visits. | 57.2% |
| OAI-SearchBot share SearchBot is the second-largest crawler in the study. | 15.1% |
| ClaudeBot share ClaudeBot is smaller in raw volume but more diverse in behavior. | 3.8% |
Population and method
Built from 575,788 AI crawler visits across 84 brands and 314,501 unique URLs observed between June 11, 2025 and February 1, 2026.
Where the evidence stops
This study measures observable crawler traffic rather than hidden fetches or internal cache reads.
One dominant e-commerce brand was excluded to keep the page-type patterns more generalizable.
Visit share and homepage preference vary sharply by bot family, so averages can hide crawler-specific behavior.
Cite this tracker.
Source study: Crawler Behavior“GPTBot dominates the crawler landscape, but the behavior of smaller bots still matters because they enter and read sites differently.”
Mack Grenfell · AI crawler traffic share by bot
When AI Comes to Your Website · Trakkr Research · Updated 1 February 2026