Google's help page defines the status as a page that appears in the index, but Google could not read its content. It names two causes: the page is cloaked to Google, or it is in a format Google cannot index. It also says robots.txt blocking is not the issue. In practice, check what Googlebot receives when it requests the URL, which can differ from what you see: an empty HTML shell, a bot-protection page, or an empty response. Then fix that response.
What the status means
The URL made it into the index, so Google found and stored it. What failed was reading the content. That makes this different from crawled, currently not indexed, where Google read the page and passed on it, and different from blocked by robots.txt, which Google says is not the cause here. See crawling vs indexing if the distinction is new.
Google does not give a reason per URL, so the table below lists the situations that fit its two documented causes and the JavaScript guidance. Treat each row as a hypothesis to test.
Cause, check, fix
| Likely cause | How to check | Fix |
|---|---|---|
| Content exists only after JavaScript runs (app-shell page) | Compare raw HTML from curl with the rendered HTML in URL Inspection. Google says content not visible in the rendered HTML cannot be indexed. | Server-side rendering or pre-rendering, so the first response already contains the text. Google recommends both. |
| Googlebot gets a different page than visitors (firewall, bot protection, CDN challenge) | Request the page with a Googlebot user-agent, then check server or CDN logs for the response sent to verified Googlebot IPs. | Allow verified Googlebot traffic. Match against Google's published IP ranges, not the user-agent string alone. |
| Cloaking, on purpose or by a plugin (content for users, something else for crawlers) | Run the live test in URL Inspection and compare the rendered page with what you see in a browser. | Serve the same content to Google and to users. Remove the rule or plugin that branches on user-agent. |
| Empty or truncated server response (200 with no body, a crash after the headers) | Use curl with a byte count and the status line. Check error logs at the request time. | Fix the application error. Return a real error status for a broken page instead of an empty 200. |
| A format Google cannot index (wrong content type, non-text file served at a page URL) | Read the content-type header from curl -I and confirm the URL returns HTML. | Serve HTML with a text/html content type, or move the file to a URL meant for it. |
| Robots meta and header rules interacting with rendering | Check for noindex in the HTML, in the rendered DOM and in the X-Robots-Tag header with the noindex checker. Google says pages with noindex may skip rendering. | Make the directives match your intent in both raw and rendered HTML. Do not add noindex in the shell and remove it with a script. |
Diagnose it in six steps
- Inspect the URL. Open URL Inspection for the exact URL. The Page indexing section reports the status. Google recommends inspecting the page when you see it.
- Test the live URL. Run the live test, then open the rendered HTML (the screenshot view, where your account shows one, helps for a quick visual check). Google's JavaScript guidance names the URL Inspection tool and the rendered HTML as the place to check what it can index.
- Compare with the raw response. Fetch the page without a browser and look for a sentence from your real content. If it is missing from the raw HTML and present in the rendered view, the page depends on rendering.
- Request as Googlebot. Fetch the page with a Googlebot user-agent. A different body or a challenge page points to bot protection or cloaking. A match does not clear you, because protection often keys on IP address too.
- Read the logs. Find requests from Google's IPs and look at the status, size and any challenge or block entries. Verify the IP by reverse DNS (the host must end in googlebot.com, google.com or googleusercontent.com, and a forward lookup must return the same IP) or by matching the ranges in Google's common-crawlers.json file.
- Retest after each change. Run the live test again. Request indexing once when the rendered HTML shows the content.
Worked example: a page that is empty before JavaScript
A launch site returns this to any client that does not run scripts:
$ curl -s https://example.com/ | head -c 400
<!doctype html>
<html><head><title>Acme</title></head>
<body><div id="root"></div>
<script src="/assets/app.4f2a.js"></script>
</body></html>
$ curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" \
-o /dev/null -w "%{http_code} %{size_download} %{content_type}\n" https://example.com/
200 312 text/html; charset=utf-8
The body is a root element and a script tag, 312 bytes in total. Google says an app shell like this needs JavaScript to reach the content, and content that never appears in the rendered HTML cannot be indexed. In a browser the page looks fine, so nobody notices. The fix is to render the page on the server or pre-render it at build time, so the first response contains the headline and body text. Then rerun the same curl and expect your visible text in the output.
If the same request with the Googlebot user-agent returns a 403, a 503 or a small interstitial instead, the problem sits in front of your application, in a firewall or CDN rule.
The AI crawler angle
Googlebot queues every page with a 200 status for rendering, so a client-rendered page can eventually be read. Other crawlers may not wait. Google's own guidance says server-side rendering and pre-rendering also help because some crawlers cannot run JavaScript. Vercel's research on AI crawlers found that none of the major AI crawlers it tested render JavaScript. They included OpenAI's, Anthropic's, Meta's, ByteDance's and Perplexity's crawlers, and Vercel noted that some fetch script files without running them.
The gap shows up on new sites. In SiteTiller's study of Show HN launches, 24 of 180 sampled sites (13.3%, 95% CI 9.1-19.1) served under 20 readable words in the homepage HTML. A page like that has little for a non-rendering crawler to read. You can check your own robots rules for AI user-agents with the AI crawler checker, and read how Cloudflare settings can block AI crawlers if your site sits behind it.
Fixes in order of effort
- Render on the server or at build time. Use the framework's server rendering, static generation or a prerender step so the HTML contains the title, headings, body text and links.
- Allow verified Googlebot. In the firewall or bot-management settings, allow requests whose IPs match Google's published ranges. Google notes that the crawler JSON files do not cover every Google IP, such as Apps Script, so use them for Googlebot checks only.
- Review CDN bot and challenge settings. Look for rules that issue a challenge, a block or an alternate page to crawlers. Review managed rules separately from your own, since they change without a deploy.
- Fix the response itself. Return a full HTML body, the right content type and an honest status code.
Common mistakes
- Trusting the user-agent. Anyone can send the Googlebot string. Allow-listing by user-agent alone opens the site to scrapers, and blocking by it can miss the actual cause.
- Looking only at the browser. A visitor's browser runs scripts, accepts cookies and passes challenges. A crawler may do none of that.
- Editing robots.txt. Google says robots.txt is not the cause of this status, so changes there will not clear it.
- Testing once. The first request and a later one can get different answers when a rate limit or reputation rule kicks in. Check logs, not just one curl.
- Clicking request indexing repeatedly. Fix the response first. A repeat request on an unchanged page returns the same result.
- Search Console Help: Page indexing report, accessed 2026-11-16.
- Google Search Central: Understand JavaScript SEO basics, accessed 2026-11-16.
- Google Search Central: Verifying Googlebot and other Google crawlers, accessed 2026-11-16.
- Vercel: The rise of the AI crawler, accessed 2026-11-16.
- SiteTiller's study of Show HN launches, accessed 2026-11-16.
Cloaking is a Google-named cause. If a plugin or edge rule shows crawlers something different from visitors, remove it even if the intent was harmless, such as a lighter page for bots.
Questions people ask
Is Page indexed without content caused by robots.txt?
No. Google's Page indexing help says this status is not about robots.txt blocking. It lists two causes: the page is cloaked to Google, or it is in a format Google cannot index. Blocked URLs have their own statuses.
Does the page rank while it has this status?
Google gives no ranking guidance for the status. Its definition says the URL is in the index but the content could not be read, so there is little text for Google to match to a query. Treat it as a defect to fix, not a state to wait out.
Can client-side JavaScript alone trigger it?
It can contribute. Google says content must appear in the rendered HTML to be indexed, and that a page whose initial HTML lacks the content depends on rendering. Compare the raw HTML with the rendered HTML in URL Inspection to see which one is empty.
Do AI crawlers see the same content as Googlebot?
Not necessarily. Vercel's research on AI crawlers found that none of the major ones it tested executed JavaScript. A page that Google renders correctly can still look empty to them.