Not sure where to start? Take a guided tour.
AI Site Grader
AI Query Scanner
llms.txt GeneratorCrawler depth and homepage entry benchmark
The benchmark data demonstrates a clear pattern of shallow crawling behavior across AI bots, where the vast majority of interactions are single-page visits rather than deep site explorations. Additionally, the point of entry varies significantly by bot family, suggesting distinct data gathering strategies between different AI models.
Research tracker page for the study When AI Comes to Your Website, detailing the crawler depth and homepage entry benchmark.
The key numbers.
- One-visit sessions
- Most AI crawler visits are one-and-done sessions.
- 88.5%
- 10+ visit sessions
- Deep repeat sessions are extremely rare.
- 0.3%
- GPTBot homepage rate
- GPTBot rarely starts on the homepage.
- 2.75%
- ClaudeBot homepage rate
- ClaudeBot is much more likely to touch the homepage.
- 19.18%
The ranked view.
- 188.5%
One-visit sessions
Single-shot crawling dominates the study.
- 219.18%
ClaudeBot homepage rate
ClaudeBot is much more likely to use the homepage.
- 32.75%
GPTBot homepage rate
GPTBot almost never uses the homepage as a launchpad.
- 40.3%
10+ visit sessions
Deep sessions are vanishingly rare.
What changed
One-visit sessions
One-visit sessions account for 88.5 percent of the recorded data, indicating that single-shot crawling dominates the study.
ClaudeBot homepage rate
The ClaudeBot homepage rate stands at 19.18 percent, showing that ClaudeBot is much more likely to use the homepage.
GPTBot homepage rate
The GPTBot homepage rate is measured at 2.75 percent, demonstrating that GPTBot almost never uses the homepage as a launchpad.
10+ visit sessions
Sessions with 10 or more visits represent just 0.3 percent of the benchmark, confirming that deep sessions are vanishingly rare.
| Measure | Result |
|---|---|
| One-visit sessions Most AI crawler visits are one-and-done sessions. | 88.5% |
| 10+ visit sessions Deep repeat sessions are extremely rare. | 0.3% |
| GPTBot homepage rate GPTBot rarely starts on the homepage. | 2.75% |
| ClaudeBot homepage rate ClaudeBot is much more likely to touch the homepage. | 19.18% |
Population and method
Built from 575,788 AI crawler visits across 84 brands and 314,501 unique URLs observed between June 11, 2025 and February 1, 2026.
Where the evidence stops
This study measures observable crawler traffic rather than hidden fetches or internal cache reads.
One dominant e-commerce brand was excluded to keep the page-type patterns more generalizable.
Visit share and homepage preference vary sharply by bot family, so averages can hide crawler-specific behavior.
Cite this tracker.
Source study: Crawler Behavior“The benchmark data demonstrates a clear pattern of shallow crawling behavior across AI bots, where the vast majority of interactions are single-page visits rather than deep site explorations. Additionally, the point of entry varies significantly by bot family, suggesting distinct data gathering strategies between different AI models.”
Mack Grenfell · Crawler depth and homepage entry benchmark
When AI Comes to Your Website · Trakkr Research · Updated 1 February 2026