AI search

Do AI crawlers run JavaScript? Data from 180 new sites

Vercel's crawler analysis says the major AI crawlers it measured do not run JavaScript. We checked what that means for real sites: 180 homepages from recent Show HN posts, fetched as raw HTML, and counted the words a crawler could read.

By Adem ÇakalPublished 4 sources6 min read
Short answer

Most do not, according to the best public measurement we found. Vercel reported in December 2024 that none of the major AI crawlers it measured, including OAI-SearchBot, GPTBot, ClaudeBot and PerplexityBot, rendered JavaScript. Google and Apple do render pages. In our own check of 180 new sites, 24 (13.3%) sent fewer than 20 readable words in the homepage HTML, so a crawler that does not run scripts would read almost nothing there.

What the primary sources say

Whether a crawler runs JavaScript decides what it can read on any site built as a client-side app. Four sources are worth reading, and they do not all cover the same bots.

CrawlerRuns JavaScript?Source and what it says
OAI-SearchBot, GPTBot, ChatGPT-UserNo, in Vercel's dataVercel's analysis lists all three as not rendering JavaScript. OpenAI's crawler page says nothing on the subject.
ClaudeBotNo, in Vercel's dataVercel adds that ChatGPT and Claude fetch script files (11.50% and 23.84% of their requests) but cannot read client-side rendered content.
PerplexityBotNo, in Vercel's dataListed with the non-rendering crawlers. It was the smallest of the five by volume (24.4 million fetches in the month measured).
Googlebot (Search and Gemini)YesGoogle: pages with a 200 status are queued for rendering in a headless Chromium. Vercel says Gemini uses Googlebot's infrastructure.
ApplebotRenders, per VercelApple says Applebot may render page content in a browser, and may fail to if script and CSS files are blocked in robots.txt.

Two cautions apply. Vercel's figures describe traffic on its own network in one month, and the article is almost two years old at the time of writing. Crawler behaviour changes without notice, so treat the table as a starting point and your own server logs as the final word. We found no equivalent statement from Anthropic, Perplexity or OpenAI in their own documentation, so we do not claim one.

Even Google, which renders, says a server-rendered response is the safer design. Its JavaScript guide recommends server-side or pre-rendering because it helps users and crawlers, and because "not all bots can run JavaScript". Rendering also happens after crawling, in a queue where a page "may stay on this queue for a few seconds, but it can take longer than that".

Our check: 180 new Show HN homepages

To see how often this matters in practice, we fetched the homepage HTML of sites posted to Show HN in September and October 2026 and counted the visible words in the response. Without running any script, we asked a simple question: how much text is there?

Visible words in homepage HTMLSitesShare95% interval
Under 20 (almost nothing readable)24 of 18013.3%9.1% to 19.1%
Under 50 (thin or empty)30 of 18016.7%11.9% to 22.8%

Put differently, roughly one new site in eight sent a homepage that a non-rendering crawler could not read, and about one in six sent a thin one. The intervals are Wilson 95% intervals; with 180 sites they are wide, so read the figures as "somewhere around one in 11 to one in 5" rather than as exact rates.

How we counted

  • Sample. From 7,833 Show HN posts with an outside link in the two-month window, we kept one host each and excluded code, social, store and document platforms by rule. That left 4,411 hosts. We took the first 200 in a fixed hash order, with no choice by hand, and the rescan counted 180 in this analysis.
  • Fetch. One GET of the homepage, raw HTML, no JavaScript executed, public pages only, robots.txt read first.
  • Count. Words in the body outside script, style, noscript, template and svg tags. Text in the header and navigation counts.
  • Manual spot checks. Our first counter flagged more sites than the final figure. We read the flagged sites one by one in their raw HTML and found false positives: text that sat inside noscript, pages with more text than the threshold, and redirects. We fixed the counter and rescanned, which produced the figures above. The sites we confirmed by hand all appear in the rescan's under-20 list.

What the numbers do not tell you

Show HN skews toward developer tools and indie software, so these are not typical small-business sites such as salons or local shops. We looked at the homepage only. The data is a snapshot from November 2026, and a young site may change its setup soon after launch. Finally, a low word count shows that the HTML is empty, not that any particular AI tool failed to cite the site.

Check your own page

The What AI crawlers see tool runs the same raw fetch on one URL and shows the status, redirect chain, word count, title, headings and first 600 characters, plus whether robots.txt allows seven crawlers. For a quick manual check, you can also run this in a terminal and read the output:

curl -sL https://example.com/ | sed -e 's/<script[^>]*>.*<\/script>//g' | head -c 2000

If you see your headline and paragraphs, the text is in the HTML. If you see an empty container and script tags, it is not.

How to fix a nearly empty homepage

  1. Find which routes matter. Home, pricing, features and docs pages carry the text you want read and quoted. Private app screens do not need this.
  2. Render those routes ahead of the request. Use server-side rendering or static generation in your framework (Next.js, Nuxt, SvelteKit, Astro and Angular all support it). For a Vite or Create React App single-page app, add a prerender step for public routes.
  3. Put the essentials in the HTML. The title, meta description, one H1 and the first paragraph should be in the server response, not created by script.
  4. Use real links. Google can only discover links written as <a> elements with an href attribute.
  5. Do not block scripts and styles in robots.txt. Apple notes that blocking them can stop Applebot from rendering a page properly.
  6. Re-run the check after each deploy. A framework upgrade or a new loading spinner can move text back into a script.

Readable HTML gets the crawler in the door, nothing more. Access is a separate question: robots.txt rules and CDN settings decide which bots can fetch the page at all. See the AI crawler checker, the guide to OpenAI's crawlers and our note on Cloudflare and AI crawlers.

Questions people ask

Do ChatGPT and Claude crawlers run JavaScript?

Vercel's December 2024 analysis of crawler traffic on its network found that OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot and Anthropic's ClaudeBot did not render JavaScript. OpenAI's own crawler page does not say either way, so check your logs if it matters to you.

Does Googlebot run JavaScript?

Yes. Google documents that it queues pages with a 200 status for rendering and runs their JavaScript in a headless Chromium once resources allow. The wait can be a few seconds or longer.

If my site is a single-page app, is it invisible to AI tools?

Not necessarily. Some crawlers render pages, and a single-page app can ship server-rendered HTML for public pages. Fetch your homepage as raw HTML and see whether your headline and body text are in it.

How many words count as readable?

In our study, under 20 visible words in the raw HTML counted as almost nothing readable and under 50 as thin. Those are our own triage thresholds, not a rule from any search engine.

Sources

  1. Vercel: The rise of the AI crawler (December 17, 2024), accessed 2026-11-16.
  2. Google Search Central: Understand JavaScript SEO basics, accessed 2026-11-16.
  3. Apple: About Applebot, accessed 2026-11-16.
  4. OpenAI: Overview of OpenAI crawlers, accessed 2026-11-16.