GPTBot

GPTBot is OpenAI’s web crawler. It can be used both to train future models and to surface content in ChatGPT search. Decide how to handle it in your robots.txt.

GPTBot is the most widely blocked AI crawler—and also the most widely used. Blocking it is a meaningful SEO decision in 2026.

How to control GPTBot

• Allow. Default. Lets OpenAI use your content for training and for ChatGPT search
• Block. Add to robots.txt:

  User-agent: GPTBot
  Disallow: /
  

• Block training only. Use OAI-SearchBot for search, block GPTBot for training:

  User-agent: GPTBot
  Disallow: /
  User-agent: OAI-SearchBot
  Allow: /
  

Why allow it

• Your content can be cited in ChatGPT and ChatGPT Search responses
• It drives AI referral traffic
• It contributes to your citation rate on the most-used AI surface

Why block it

• You do not want your content used for training future OpenAI models
• You have a paywall or licensed content that should not be re-stated
• You are in a regulated industry with content restrictions

How to verify

• Check your server logs for User-Agent: GPTBot and User-Agent: OAI-SearchBot
• Use OpenAI’s published IP ranges to confirm
• Test by asking ChatGPT whether it can find your newest page