OAI-SearchBot decides whether you can appear in ChatGPT search answers. GPTBot collects pages that may train OpenAI's models. ChatGPT-User fetches a page when a person asks ChatGPT to, and robots.txt may not apply to it. OAI-AdsBot only visits pages submitted as ChatGPT ads. Each setting is independent, so "in ChatGPT search, out of training" is one robots.txt group away.
The four bots side by side
| OAI-SearchBot | GPTBot | ChatGPT-User | OAI-AdsBot | |
|---|---|---|---|---|
| Job | Surfaces sites in ChatGPT search | Crawls content that may train foundation models | User actions in ChatGPT and custom GPTs | Checks pages submitted as ads |
| Crawls on its own? | Yes | Yes | No, only when a user triggers it | Only submitted ad pages |
| Follows robots.txt? | Yes | Yes | "May not apply" | Not stated |
| If you block it | Not shown in ChatGPT search answers; can still appear as a navigational link | Content signalled as not for training | Firewall needed to stop it | Only matters if you run ChatGPT ads |
| IP list | openai.com/searchbot.json | openai.com/gptbot.json | openai.com/chatgpt-user.json | openai.com/adsbot.json |
One detail is easy to miss. When a site allows both OAI-SearchBot and GPTBot, OpenAI says it may use the results of one crawl for both purposes to avoid crawling twice. Your logs may then show fewer GPTBot visits than you expect, even though training access is open.
What "opted out of search" actually removes
robots.txt for the three common choices
In ChatGPT search, out of training
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /
Out of ChatGPT entirely (as far as robots.txt reaches)
User-agent: OAI-SearchBot
User-agent: GPTBot
User-agent: ChatGPT-User
Disallow: /
ChatGPT-User may still fetch pages a user asks about, so the last line is a request, not a lock.
Open a docs section to search, keep the rest closed
User-agent: OAI-SearchBot
Allow: /docs/
Disallow: /
The longer matching rule wins, so /docs/ pages are allowed and everything else is not. Test any of these against your live file with the AI crawler checker, adding a path such as /docs/ to the address.
Finding OpenAI's bots in your logs
Each bot names itself in the user-agent string, with a version number that OpenAI says may change. Match on the token, not the full string:
# access log lines, simplified
"GET /pricing HTTP/1.1" 200 "... compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot"
"GET /blog/setup HTTP/1.1" 200 "... compatible; GPTBot/1.4; +https://openai.com/gptbot"
"GET /blog/setup HTTP/1.1" 200 "... compatible; ChatGPT-User/1.0; +https://openai.com/bot"
# count visits per bot (Linux, combined log format)
grep -oE "OAI-SearchBot|GPTBot|ChatGPT-User|OAI-AdsBot" access.log | sort | uniq -c
Anyone can put "GPTBot" in a user agent. To know a visit is really OpenAI, check the request IP against the matching JSON list above. A scraper using OpenAI's name from a residential IP is not OpenAI, and blocking it does not affect ChatGPT.
If your host or CDN shows "verified bot" labels, that check has usually been done for you. If you see ChatGPT-User hits on a page, it often means someone pasted that page into ChatGPT, which is a signal worth noticing on its own.
What none of these settings control
- Whether you are cited. Allowing OAI-SearchBot makes you eligible. Being chosen depends on whether your page answers the question better than the alternatives.
- Content already collected. OpenAI's page describes the GPTBot block as a signal about use in training; it does not say what happens to content gathered before you added it.
- Other companies' crawlers. Claude, Perplexity, Google and Bing each have their own tokens. The checker covers 13 of them in one pass.
- Firewall blocks. A CDN rule can stop OpenAI's bots even when robots.txt allows them; see Cloudflare and AI crawlers.
Questions people ask
If I block GPTBot, will ChatGPT still cite my site?
It can. OpenAI describes GPTBot (training) and OAI-SearchBot (search) as independent settings. Block GPTBot, allow OAI-SearchBot, and your pages can still be shown in ChatGPT search answers.
How long until a robots.txt change affects ChatGPT search?
OpenAI says it can take about 24 hours after a robots.txt update for its search systems to adjust.
Why does ChatGPT-User visit pages I disallowed?
ChatGPT-User acts on a person's request inside ChatGPT, for example when they paste your link. OpenAI says robots.txt rules may not apply to these user-initiated actions. If you must stop it, block its published IP ranges at your firewall.
Is a visit from OAI-SearchBot proof that I rank in ChatGPT?
No. It means your page can be considered for ChatGPT search. Whether it is cited depends on the question and the competing pages. Track citations separately.
Related
Sources
- OpenAI: Overview of OpenAI crawlers, accessed 2026-11-12.
- OpenAI: OAI-SearchBot IP ranges (searchbot.json), accessed 2026-11-12.
- OpenAI: GPTBot IP ranges (gptbot.json), accessed 2026-11-12.
- OpenAI: ChatGPT-User IP ranges (chatgpt-user.json), accessed 2026-11-12.