GPTBot: what it is and how to allow or block it
OpenAI’s crawler for collecting pages that may be used to train its AI models.
Last checked 2026-10-08
- Company
- OpenAI
- Used for
- OpenAI model training
- Job
- Collects pages to train AI models
- Follows robots.txt
- Yes
- Name in robots.txt
- GPTBot
What GPTBot does
GPTBot is the crawler OpenAI uses to gather pages for training. OpenAI describes it as being used “to make our generative AI foundation models more useful and safe.”
It has nothing to do with whether ChatGPT can find and link to your pages in its search answers. That job belongs to a different bot, OAI-SearchBot, which you control separately.
Should you allow it?
A genuine choice, and it depends on how you feel about your content being used for training. Blocking it costs you nothing in ChatGPT search, so many businesses block GPTBot while allowing OAI-SearchBot.
What blocking it changes: Your pages should no longer be collected for training future OpenAI models. It does not remove you from ChatGPT search results, which are controlled by OAI-SearchBot.
Allow or block GPTBot in robots.txt
OpenAI says that disallowing GPTBot “indicates a site’s content should not be used in training generative AI foundation models.”
To allow it
User-agent: GPTBot Allow: /
To block it
User-agent: GPTBot Disallow: /
robots.txt lets every bot in by default, so you only need the allow lines if another rule in your file would otherwise block it. Our robots.txt guide explains how rules for different bots combine.
How it appears in your server logs
OpenAI publishes this as the bot’s user-agent string, the label it sends with every visit:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
How to check a visit is really GPTBot
OpenAI publishes the IP addresses GPTBot crawls from. A visit claiming to be GPTBot from any other address is not GPTBot.
Our guide to server log checks shows where to find these visits on common hosting setups.
Source: OpenAI crawler documentation, checked 2026-10-08. Bots and their rules change often, so check the company’s own page before making a decision that matters.
See which AI bots your site lets in
The free AI readiness check reads your robots.txt and reports which of the main AI crawlers are allowed or blocked, along with everything else that decides whether AI can understand your site.
Check your site free