OpenAI runs three crawlers, and treating them as one is the most common robots.txt mistake. Each is controlled by its own token (verified in our crawler directory).
The three, plainly
- GPTBot — training. Content it fetches may be used to train OpenAI's models.
- OAI-SearchBot — search. Surfaces your site in ChatGPT's search results. This is the one that brings referral visibility.
- ChatGPT-User — user fetch. Retrieves a page when a ChatGPT user explicitly asks about it.
The usual goal: be findable, not trained on
Many owners want to appear in ChatGPT's answers while opting out of training. Per OpenAI's documentation that's: disallow GPTBot, allow OAI-SearchBot and ChatGPT-User. Blocking GPTBot does not reduce your ChatGPT search visibility — different bot. Check your live setup with the crawler check.