What is OAI-SearchBot? AI crawler guide
OAI-SearchBot is a crawler or fetcher associated with OpenAI. See source-graded user-agent, robots.txt, verification, and observation guidance.
What is OAI-SearchBot?
OAI-SearchBot is a documented OpenAI crawler or fetcher. OpenAI search crawler for indexing pages that can appear in ChatGPT search results.
Evidence status
| Field | Value |
|---|---|
| Evidence | Officially documented |
| Lifecycle | Active |
| Purpose | search |
| robots.txt posture | honors |
| Source checked | 2026-08-18 |
Documented user-agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Allowing OAI-SearchBot creates the possibility of retrieval. To find out whether ChatGPT actually cites or recommends you, track your ChatGPT mentions over time.
robots.txt allow example
User-agent: OAI-SearchBot Allow: /
robots.txt block example
User-agent: OAI-SearchBot Disallow: /
What the rule can and cannot do
Robots.txt expresses an access policy to compliant automated crawlers. It does not authenticate the sender, remove content already collected, or guarantee that a model will use or not use content.
How to verify a request
- Match the complete documented user-agent where one exists. A match is only a clue.
- Use the operator-published check: Published IP ranges.
- Keep the observed request, operator documentation, and any inference as separate fields.
Observed in Trakkr connected-site data
| Measure | Value |
|---|---|
| Matching signature | OAI-SearchBot |
| Classified requests | 354209 |
| Sites in sample | 85 |
| Window | 2026-07-18 through 2026-08-17 |
| Method | Finalized daily crawler summaries from connected Trakkr sites, grouped by the crawler signature detected in each request. |
| Limits | This is a connected-site sample, not a representative sample of the web. A matching user-agent or signature does not prove that the named operator sent the request. Counts describe classified request signatures, not market share or unique pages crawled. |
JavaScript behavior
The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.
Related bots
- PerplexityBot: Also tracked as a search crawler.
- Kimi-SearchBot: Also tracked as a search crawler.
- Amzn-SearchBot: Also tracked as a search crawler.
- MistralAI-Index: Also tracked as a search crawler.
- ExaSearchBot: Also tracked as a search crawler.
- Kagibot: Also tracked as a search crawler.
- Claude-SearchBot: Also tracked as a search crawler.
- YouBot: Also tracked as a search crawler.
- Googlebot: Also tracked as a search crawler.
- SearchGPT: OAI-SearchBot is a search crawler tied to AI answer visibility.
- ChatGPT: OAI-SearchBot gives crawler context for ChatGPT.
Frequently Asked Questions
What is OAI-SearchBot?
OAI-SearchBot is a officially documented crawler or fetcher record associated with OpenAI.
What user-agent does OAI-SearchBot use?
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot. Check the evidence status before attributing a matching request.
Can OAI-SearchBot be blocked in robots.txt?
The record identifies OAI-SearchBot as the token to review. Robots.txt is a request policy, not proof of model use or non-use.
How can I verify a OAI-SearchBot request?
Start with the full user-agent, then use published ip ranges from the operator. A user-agent match alone is not proof.
Data & Sources
- OpenAI documentation - Primary source for OAI-SearchBot crawler details.
- OAI-SearchBot verification: Published IP ranges - Verification information published for OAI-SearchBot.