Learn

Crawling vs indexing vs ranking: where your page is stuck

A page that is missing from Google can be stuck at three different points, and each point has different fixes. This guide follows Google's own description of the pipeline and maps every Page indexing status to the stage it belongs to, so you can go straight to the right fix.

By Adem ÇakalPublished 3 sources6 min read
Short answer

Crawling is Google fetching the page. Indexing is Google analysing and storing it. Serving is Google choosing it for a query. A page can pass one stage and fail the next, and Google says it does not guarantee any of them. Find the stage first using the status in Search Console, then apply the fix for that stage only.

The pipeline in four steps

Google's overview describes crawling, indexing and serving. Between the first two, Google renders the page, so we show four boxes. Google has no central registry of pages, so it finds URLs by following links and reading submitted sitemaps.

Four stages: crawl, render, index, serve, with the typical blockers under each 1. Crawl Googlebot fetches the URL 2. Render Runs JavaScript in a recent Chrome 3. Index Analyse, cluster, pick canonical, store 4. Serve Match to a query, choose and rank robots.txt, 4xx, 5xx, redirect errors, login Content only visible after scripts run noindex, duplicates, soft 404, not chosen Relevance, quality, location, language
Stages follow Google's "How Search works" page; the blocker lists summarise the Page indexing report. Checked November 12, 2026.

Page indexing statuses mapped to stages

Names below are as written in Google's Page indexing help. The first stage that fails is the one to fix.

Status in Search ConsoleStageWhat to do next
URL blocked by robots.txtCrawlBlocked by robots.txt; test rules in the robots.txt tester
Server error (5xx), Redirect errorCrawlFix the server or the redirect chain, then re-test with URL Inspection
Not found (404), 401, 403, other 4xxCrawlRestore the page or accept that it is gone; for 401 and 403 let Googlebot through
Discovered, currently not indexedBetween crawl and indexGoogle knows the URL but has not crawled it: fix page
Crawled, currently not indexedIndexFetched but not stored: fix page
URL marked noindexIndexnoindex checker; if you removed it, noindex stuck
Soft 404IndexReturn a real 404, or add real content to the page
Duplicate without user-selected canonical; Duplicate, Google chose different canonical than userIndex (canonical choice)duplicate canonical fix
Alternate page with proper canonical tagIndex (canonical choice)Usually intended; nothing to fix
Page with redirectIndex (redirect target indexed instead)fix page
Indexed, though blocked by robots.txtIndex (warning)Decide whether you want it indexed; see noindex vs robots.txt vs canonical
Page is indexedServeIndexing is done. For no traffic, read indexed but not showing

How to use the table

  1. Open the URL Inspection tool for the page and read the status. Our field-by-field guide explains the sections.
  2. Match the status to a row. If the row says Crawl, no indexing change will help until Google can fetch the page.
  3. If the status is Page is indexed but the page gets no visits, you are in the serving stage, which is about relevance and competition, not access.
  4. Check that the page is listed in your sitemap. Sitemaps help discovery; Google does not promise to crawl everything in them.

Two points that cause confusion

Rendering matters for scripts. Google says it runs JavaScript during the crawl using a recent version of Chrome. If your main text appears only after scripts run, a rendering problem looks like a thin page at the indexing stage, so test the live URL and view the rendered HTML.

Not indexed is sometimes fine. Google's help describes some statuses, such as an alternate page with a proper canonical, as expected behaviour. Fix only the URLs that you want to appear in search.

Worked example: a new blog post that never appears

Say you published a post two weeks ago and a search for its exact title returns nothing. Open URL Inspection and read the status. If it says Discovered, currently not indexed, Google has the address but has not fetched it, so no change to the page content will matter yet. Strengthen internal links to the post and confirm it is in the sitemap. If it says Crawled, currently not indexed, Google has read the page and chose not to store it, so now the content and its overlap with other pages matter. If it says URL marked noindex, a template or plugin is adding the directive, and the page itself is fine. Each status leads to a different task, which is the reason to read the status before changing anything.

Keep a note of the date you inspected, because statuses change as Google recrawls. A page that was Discovered last week may be Crawled this week, and that is progress, not a new problem. Re-test with the live URL option after each fix rather than waiting for the report to refresh.

Questions people ask

Is a crawled page always indexed?

No. Google's documentation says indexing is not guaranteed and not every processed page is indexed. The status Crawled, currently not indexed exists for exactly this case.

Does robots.txt keep a page out of the index?

It controls crawling. Google lists Indexed, though blocked by robots.txt as a warning status, meaning a blocked URL can still be indexed. To keep a page out of results, use noindex on a page Google is allowed to fetch.

Is an indexed page guaranteed to rank?

No. Google says a page can be indexed and still not appear in results, for example when the content is judged irrelevant or low quality, or when robots meta rules prevent serving.

Which tool shows the stage for one URL?

The URL Inspection tool shows discovery, crawl and indexing details per URL. Our walkthrough of its fields is linked in the Related list.

Sources

  1. Google Search Central: How Google Search works, accessed 2026-11-12.
  2. Search Console Help: Page indexing report, accessed 2026-11-12.
  3. Search Console Help: URL Inspection tool, accessed 2026-11-12.