We get asked this constantly: "should I just turn on JavaScript rendering for everything?" No. Rendering (loading a page in a real browser engine and crawling the DOM it produces after scripts run, instead of the raw HTML response) is slower and heavier per page, and for the large majority of sites it changes nothing about what gets found. Most frameworks — Next.js, Nuxt, SvelteKit, plain server-rendered PHP or Rails, WordPress — either render on the server or progressively enhance a page that already has its content and links in the initial response. The raw HTML is the real page, not a shell.
The exception is real, though, and worth taking seriously when it applies: client-side-rendered single-page apps (SPAs) — a React, Vue, or Angular build that ships a near-empty HTML document and builds the entire page in the browser via JavaScript after load. Crawl one of those without rendering and you get a blank shell: no title worth reading, no body text, no internal links to follow. The crawl doesn't fail loudly — it just quietly reports a site with nothing on it, which is worse, because nothing tells you to double-check.
Check view-source before you check a setting
You don't need a tool to answer this — your browser already has the two views you need, and comparing them takes under a minute per template.
- View source (
Ctrl+U/Cmd+Option+U), which shows the raw HTML exactly as it arrived over the wire, before any script has run. - Inspect element (
F12/Cmd+Option+I, Elements panel), which shows the live DOM after the browser has executed every script.
Load the page, open both, and compare. Is your actual heading text, body copy, and the internal links a user would click already sitting in view-source? Then the raw HTML is the page — a text-only crawl sees exactly what a user (and a search engine) sees, and rendering would cost you time and machine resources for identical results. Does view-source show little more than an empty container — something like <div id="root"></div> or <div id="app"></div> — with everything else built by inspect element only after scripts run? That's the shell pattern, and it's the case rendering exists for.
<a href> links; content matches what you see rendered; "Disable JavaScript" in dev tools still shows the page's core content and nav<div>, a script tag, little else); links and text only appear in inspect element; "Disable JavaScript" leaves a blank or broken pageThat last row matters more than it looks. We've seen sites where the marketing pages and blog are fully server-rendered — plain HTML, fast, fine as-is — while the product catalog or search results are a client-rendered widget bolted on later. Testing only the homepage and generalizing to the whole domain is the most common mistake here; check the templates that actually carry your content and internal links, not just the one that loads first.
The cost side, honestly
Rendering isn't free, and we'd rather say that plainly than sell you on flipping a switch with no downside. A real headless-browser render (RacingRoach's optional Chromium/CDP backend — see how the engine's fetch layer works) has to load the page, run its scripts, wait for network requests to settle, then hand off the resulting DOM — meaningfully more CPU and wall-clock time per page than pulling raw bytes and streaming them through a parser. That cost multiplies across every URL in the crawl, which is exactly the crawl-budget arithmetic we walked through in why response time is a crawl-budget problem: a slower per-page cost means fewer pages covered in the same window, whether that window is a search engine's daily allocation or your own patience waiting for a crawl to finish.
None of that is an argument against rendering when you need it — it's an argument against reaching for it by default. If your content and links are only reachable post-render, a slower, heavier crawl that actually sees your site beats a fast one that sees an empty div. The point is to make that call deliberately, template by template, instead of switching rendering on for every crawl because the site "feels" modern.
How to decide, in order
- View-source your key templates — homepage, a category/listing page, a detail page, and anything with an app-like feel (dashboards, search, filters). Look for your real content and internal links, not just a document.
- Disable JavaScript in dev tools and reload. If the page still shows its core text and navigation, the raw HTML carries the content — you don't need rendering for that template.
- If a template fails both checks — empty view-source, breaks with JS disabled — that's the one to crawl rendered. RacingRoach's rendering setting is a per-crawl toggle, so you can run the bulk of a mixed site text-only and point a separate, rendered crawl at just the client-rendered sections.
- Re-check after a redesign or framework migration. A site that was server-rendered last year and moved to a client-rendered SPA this year needs this decision revisited — the rendering requirement isn't a property of the domain, it's a property of the current template.
Default to text-only. Turn rendering on for the specific templates that prove they need it, not for the whole crawl because one section of the site happens to be a SPA.