GPTBot

Training crawler

OpenAI's crawler for pages that may be used to train its models. Seen on 107 of 135 sites in 13 weeks.

GPTBot reached 60 of 92 sites last week

Sites that got at least one GPTBot request each week, against every site reporting that week.

55%53%67%65%59%56%55%75%68%54%58%55%65%

Reach67%3rd of 2670 of 104 sitesLast 4 weeks

Requests2.3M6% of all crawler requests13 weeks, 107 sites

Per site352a weekMedian where it visitsLast 4 weeks

Pages a visit60.5ClaudeBot reads 5.1Snapshot, 1 Feb 2026

How GPTBot reads a site

329,572 GPTBot visits to 59 brand sites. Snapshot, 1 Feb 2026.

Depth

52%

of its visits land three folders deep, like /blog/2024/post. Only 3% touch the homepage.

When

+29%

more visits on a weekend day than a weekday. Busiest hour 14:00 UTC, but no hour is quiet.

Blocking GPTBot is a training choice. ChatGPT search uses OAI-SearchBot.

OpenAI says GPTBot follows robots.txt. A rule changes future crawls, not pages it already collected.

Block GPTBot

User-agent: GPTBot
Disallow: /

Allow GPTBot

User-agent: GPTBot
Allow: /

User agentMozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

Is it really GPTBot?

Anyone can send the GPTBot name. Requests we could check against OpenAI's addresses, 1 Aug to 14 Oct 2026.

Fakes

5.7%

of requests signed GPTBot did not come from OpenAI's published addresses.

Verified, 94.3%Fake, 5.7%

How to check

1Match the IP to OpenAI's listOpenAI publishes GPTBot's addresses at openai.com/gptbot.json.

2Read the user agent closelyOne common fake opens a bracket after Mozilla/5.0 and never closes it.

See every fake crawler

Common questions

What is GPTBot?

GPTBot is OpenAI's training crawler. OpenAI crawler for content that may be used to improve generative AI foundation models.

How do I block GPTBot?

Add a directive to your robots.txt: "User-agent: GPTBot" followed by "Disallow: /". OpenAI says GPTBot follows robots.txt. A rule changes future crawls, not pages it already collected.

Does blocking GPTBot hide me from AI?

No single bot rule hides a site from every AI product. Disallowing GPTBot expresses an opt-out for the potential training use covered by that crawler. ChatGPT search uses the separate OAI-SearchBot control.