Changelog
What's new in RacingRoach.
Every release, in plain language. For the full commit-level history, see the project's public changelog file.
Give an AI agent read access to your crawl
Racingroach now speaks the Model Context Protocol. Point Claude, or any
MCP-capable agent, at an open crawl and it can read the overview, issues, pages,
links, site structure and keywords through nine read-only tools — no writing, no
starting or stopping crawls, no settings changes. Run it from the CLI against a
saved .redb file, or expose the crawl the desktop app currently has
open.
Deeper headings, per-link status, and cleaner duplicate views
The Elements screen now captures every heading level from H1 to H6, not just H1/H2, and exports all six. The Links screen shows each link's HTTP status inline, duplicate-content findings are grouped so you see one row per duplicate set rather than a flat list, and the Resources screen lets you click any image to see which pages reference it. You can also save an in-memory crawl to disk at any point, with a banner that keeps the "not yet saved" state visible for the whole run.
Edited SERP metadata, everywhere it should be
Title and meta-description edits you make in the SERP preview now show up in the normal (non-editing) view too, and a dedicated export gives you just the pages you actually changed.
Bring your own embedding model
Internal link opportunities no longer need a bundled ~488 MB local model. Point Settings → AI at any OpenAI-compatible embeddings endpoint, or a local Ollama server if you want the whole analysis to stay on your machine.
Honest handling of failed fetches and unknown sizes
A URL that returns no response is now a real finding — a Critical "Fetch failed" issue with the underlying error — instead of a silent pass that used to raise your health score. Reopening an older saved crawl that had failures will show a lower, correct score, no re-crawl needed. Pages whose size was never measured stop matching size filters as if they were tiny, the dashboard paints no-response pages as failures rather than a green bar labelled 0, and the total-size figure is marked as a lower bound when any size is unknown. You can also retry failed pages into a running or paused crawl — a successful retry replaces the original row in place, with no duplicate pages or double-counted totals.
Export a PDF crawl report
Turn a crawl into a formatted document you can hand to a client, from one button on the Overview screen. It carries the health score and its twelve theme scores, the severity picture, the status and content-type breakdowns, the four crawl-distribution charts, and an appendix of the URLs behind every issue that fired — each section stating the real total, so a top-ten sample never reads as the whole picture. The report is self-contained, renders offline, and is free: an unlicensed export is a correct free-tier report rather than a locked one. If no browser is available to print it, the same report is saved as printable HTML and the app tells you which format it wrote.
Submit your URLs to IndexNow (premium)
Tell Bing, Yandex and the AI answer engines built on their indexes about new or changed pages, instead of waiting for their next visit. Publish your IndexNow key at your site root and verify it, then pick a source — everything changed since the site's previous run, every sitemap-eligible page, or a filtered set handed over from the Pages screen. You always review the exact list before anything is sent, URLs belonging to another host are shown as skipped, and a still-validating response is reported as its own state rather than as a completed submission. Submission is manual: nothing is ever sent without you asking.
Clean up old crawl runs
Reclaim the disk your old crawls are sitting on without leaving the app. Set an optional retention policy — keep the newest N runs, keep the last N days, or both — then use Clean up old runs on any site in Site History. You see exactly which runs would go, their paths and sizes, and the total you would reclaim, and nothing is deleted until you confirm that list. Removing a run takes its database, its screenshot and archive files, and its history row together. The crawl you have open is never deletable.
Post-crawl insights digest (premium)
Stop cross-referencing three screens after a run: the new premium Insights screen gives you one ranked, plain-language summary of what changed since the site's last crawl — what broke, what got fixed, and what to look at first — re-scored under your current thresholds and license so a settings change can never look like a regression, with one click through to the full Compare screen for the same pair.
Third-party & cross-origin inventory
See every other domain your site depends on: a new free Third Party screen lists scripts, stylesheets, media and frames loaded from outside your own site, which of them set a cookie, which scripts ship with no integrity protection, and the full cookie inventory the crawl has always observed — plus two new checks for unpinned third-party scripts and pages leaning on too many outside domains.
Site history: every crawl of a site, on one timeline
Crawls now group themselves into sites, so the Dashboard lists every site you have crawled and lets you reopen any past run with one click instead of a file dialog — with a trend line across the series, failed runs included, and a shortcut to compare the open crawl against the run before it.
Documents report for PDFs and non-HTML files
See every PDF, Word, Excel and other document your crawl found — with its size, title, PDF metadata and linking pages — in a filterable, exportable screen, plus two new checks for untitled PDFs and oversized documents.
Outbound Risk: where your site sends its readers
The new free Outbound Risk screen groups every external link in a crawl into one row per domain — with the links, the pages carrying them, how many are nofollow, and which of nine risk flags fired (a dead or unresolvable target, a redirect, credentials in the URL, an IP-literal or Punycode host, a known shortener, a wrapped URL, or anchor text naming a domain the link doesn't actually go to) — so you can see at a glance which domains you link out to and which of them are a problem.
Bring your own key: AI-suggested titles and meta descriptions (premium)
racingroach can now talk to a language model — yours, with your own API key — and the first thing it does with one is rewrite metadata. Point it at OpenAI, Anthropic, Gemini, Ollama, or any OpenAI-compatible gateway in Settings → AI, then on the SERP Preview screen hit Suggest for a single page, or run a batch over the failing pages loaded there: it names the exact number of calls before spending anything, shows live progress, and can be cancelled. Suggestions are drafts for your review — they land in the same session-only edit store as a hand-typed rewrite, so Revert and the revised-metadata CSV export work unchanged, and if a run stops partway the suggestions you already paid for are kept along with the reason it stopped. Your key lives in the OS keychain, never in a settings file, so the shareable config bundle cannot leak it — there is nothing in it to leak. A local Ollama endpoint on 127.0.0.1 is reachable while cloud-metadata and every other private address stays blocked. And because crawled page content goes into the prompt, model output is treated as untrusted input: the parser accepts only the declared shape, rejects prose rather than guessing, and strips invisible characters. Everything here is opt-in and off by default — configure no provider and racingroach makes no model request, ever.
Resource inventory: which pages load this file, and is it healthy?
racingroach already read every page's referenced stylesheets, scripts and media while crawling — and then threw the list away, so the one question you actually ask about an asset had no answer. The new free Resources screen keeps it: four tabs (stylesheets, scripts, media, images), one row per distinct file with its kind, how many pages reference it, and its status, content type and size — click any row for the pages carrying it, filter by URL, and export the whole crawl to CSV or the all-in-one workbook. Capture is unconditional and unscoped, so CDN- and third-party-hosted assets are inventoried alongside your own; those are usually the interesting rows and they're never crawled. Two free checks come with it, taking the catalog to 110: a referenced asset that fails to load, and one over a size threshold you can change without re-crawling. Crucially, absence is reported as absence — measuring status and size needs resource crawling, which stays off by default, so when nothing was measured the checks say not evaluated rather than "0 problems", they leave your health score alone, and the screen tells you how to turn measurement on. A crawl that checked nothing will never claim your assets are fine.
Missing H1 and missing favicon checks
Two checks every competing crawler reports and racingroach didn't, taking the catalog to 108. Both are free. Missing <h1> is the obvious gap: we already flagged pages with several <h1>s and pages sharing the same one, but nothing fired when a page had none at all — and because the heading count was already stored for every page, reopening an existing crawl reports this straight away with no re-crawl. No declared favicon catches the small branding detail a browser tab, a bookmark and a Google result all show beside your page. It's honest about its scope: it reads your <link rel="icon"> declaration and nothing else — it never downloads the icon and never probes the root /favicon.ico — so a site relying on that convention without declaring it in HTML is reported as missing one. That's why the wording says the page declares no favicon rather than that it has none. Favicon state is newly recorded, so this check needs a fresh crawl; reopen an older one and it correctly reports nothing rather than flagging every page.
A health score that survives one bad check
The health score on your dashboard could be dragged to zero by a single issue firing across most of your pages — one critical fault on a tenth of the site was enough — which meant the number couldn't tell a genuinely broken site apart from a decent one with a single widespread fault, and every check we add made it worse. It now scores pages rather than checks: each page accumulates the severity of what's actually wrong with it, capped at "one critical fault", and your score is the average page's health. So a site where every page returns a server error still scores 0, as it should — but one widespread warning now costs 30 points, not 100, and adding checks in future releases can't quietly deflate the number. Nothing was re-crawled and no setting changed: reopen an existing crawl and it simply reports a higher, more honest score than it did before.
Can the answer engines read you? (premium)
ChatGPT, Perplexity, Claude and Google's AI Overviews read the web through their own crawlers, and until now you had no way to check what they see. The new AI Search screen answers three things: whether your robots.txt lets each of twelve named AI crawlers in — GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest — with the operator, what the crawler is used for, and the exact line that decided the verdict; whether you've published the llms.txt files they look for; and how well each page is shaped for a passage to be lifted out of it and cited. That last part is a per-page AI citability score from 0 to 100, sortable on the Pages screen, and it is never shown without the six pass/fail facts behind it — answer-ready structure, structured data, a freshness date, extractable substance, readability, and retrievability. Five new checks (106 in total now) flag the individually fixable ones, and the whole thing exports to CSV, Excel, and a tab in the all-in-one workbook. Two honest limits: the score is our own weighted heuristic, not a published formula or a measurement, and racingroach does not query the answer engines to see whether they actually cite you — it audits readiness from your own markup, entirely offline.
What is this site actually about?
Every other report in racingroach tells you what's broken. The new free Keywords screen tells you what your site talks about: one row per term the crawl actually uses — single words, two- and three-word phrases — with how many pages carry it and how often it occurs. Click any term to see which of your own pages compete for it, which is the honest, entirely local half of what other tools sell as keyword cannibalization. A stop-word list you control keeps the report from being topped by "of the": add words, remove the built-in ones, and because the counting happens when you open the report rather than when you crawl, editing the list re-counts an existing crawl instantly with no re-crawl. Page detail gains a top-keywords panel for a single page, and the table exports to CSV or Excel and turns up as a tab in the all-in-one workbook. It works on any crawl you reopen — the text was already stored, so nothing needs crawling again.
Which section of the site is worst?
You can now answer that by sorting a table. The new free Site Structure screen shows one row per folder — page count, indexable share, mean Link Score, the status-code and content-type split, and issue counts by severity — and each row covers everything beneath that folder at any depth, so /blog tells you the state of the whole blog rather than of the two pages sitting literally in it. Click a folder to go deeper, use the breadcrumb to come back up, and the pages that stop exactly at the folder you're looking at get their own row so nothing is hidden. The level you're on exports to CSV or Excel, and it turns up as a tab in the all-in-one workbook. Nothing needs re-crawling: the report is worked out when you open it, from data your crawls already hold, so it works on anything you reopen.
Find the pages nobody can read
Every English page in a crawl now carries a Flesch reading-ease score and a grade level, free — sortable and filterable in the pages table, shown in page detail, charted as a five-band distribution, and included in your export. A new Hard to read check flags pages below a reading-ease floor you set yourself, and because it's evaluated when you look rather than when you crawl, you can move the threshold and re-count an existing crawl without re-crawling. Pages we can't score honestly — a page declaring a non-English language, a redirect, something too short to measure — report no score rather than a confident wrong one, and stay visibly distinct from pages that genuinely scored badly. The whole app also became keyboard- and screen-reader-navigable in this release: real landmarks and headings on every screen, visible focus rings, and crawl progress announced as it happens.
Your whole issue list, in one click
Getting your findings out used to mean working the Issues screen one check at a time — pick a check, export, repeat, up to a hundred times into a hundred unrelated files. There's now an Export all issues (CSV bundle) button on the Overview screen, and it's free. One click writes a dated folder holding a _summary.csv plus one findings CSV per check that actually found something, so you get findings instead of ninety empty files. The summary lists each check's severity, theme and affected-URL count alongside the same "why it matters" and "how to fix" guidance the Issues screen shows you — in whichever of the nine languages you're using — which turns a list of 4,000 URLs into something you can hand to a developer and have acted on. Each per-kind file comes out of the same export the Issues screen's own CSV button uses, so it can never disagree with what's on your screen, and the column headers and filenames stay in English so the files stay easy to script against.
Crawl trees you can actually read
The tree views were drawing every node at a fixed spacing with a label on each, which on a real crawl meant a diagram hundreds of thousands of pixels wide in a box that couldn't scroll or zoom — most of your site was simply out of reach. The trees now grow left to right so labels sit beside their node instead of on top of each other, and you can pan, zoom, fit and reset. Big crawls open partly collapsed, showing how many pages each closed branch hides, so you start with something readable and open only the parts you care about; small crawls still show everything at once, exactly as before. Clicking any node opens that page's details, and the legend finally does something: pick a status code or content type and the matching pages light up while everything else fades — the site structure stays on screen, so you can see where your 404s live rather than just how many there are. Response-time labels no longer overlap, and content types are grouped into eight families, so text/html and text/html; charset=utf-8 stop showing up as two different colours. The Force Graph has been removed — the trees do its job better.
Findings that tell you what to fix
A finding that says "3 uncrawlable links" and nothing else can't be acted on. Now every one of them names the thing that's wrong: an uncrawlable link shows the actual href or onclick it found and why the crawler can't follow it, a JavaScript error shows the error message with its source file and line, a spelling or grammar finding lists the flagged words and their suggestions, and a broken bookmark shows the target page and the #fragment that doesn't exist on it — and it all works on crawls you've already saved, without re-crawling. Page checks have also stopped firing on things that aren't pages: your stylesheets, scripts, images and PDFs no longer get flagged for a missing title or an uppercase filename, and non-HTML files aren't downloaded just to measure them, so crawls use less of your bandwidth. The archive export now refuses with an explanation when there's nothing captured to export, instead of silently handing you an empty zip. One thing to know: because junk findings no longer count against it, your health score will look different from before — the number is measured over the pages that can actually carry a problem.
One number for how healthy your site is
A crawl used to hand you a flat list of a hundred checks and leave you to read all of it. Now the Overview opens with a single health score out of 100, and the hundred checks are grouped into ten themes — Crawlability, Indexability, Content, Internal linking, Performance, Markup, Security, International, Mobile & accessibility, and URL quality — each with its own score, so you can see at a glance which part of the site is worst and start there. The score weighs how serious a problem is and how much of the site it touches, so one broken page on a big site barely moves it while the same problem everywhere drops it hard. Notices you've chosen to live with, like deliberate noindex pages, never count against you. On the Issues screen you can switch between grouping by severity and by theme, and the scores travel into the Excel workbook, the Looker Studio report and the command line. Free, and it works on crawls you've already saved.
Find the internal links you're missing (premium)
racingroach already shows the links your site has. Now it suggests the ones it should have. For every page, it finds other pages that cover related ground but don't link to it yet, and tells you which words to turn into the link and which sentence to put it in — so you can go straight to the page and make the edit. It reads meaning rather than matching keywords, so "canine nutrition" still finds the page about dog food, and it works on sites in any language. Everything runs on your own machine with no uploads and no API calls. Filter by page or by how closely the pages match, and export the lot to CSV or Excel.
Turn a crawl into a Looker Studio dashboard (premium)
Export a finished crawl straight into Looker Studio. Pick a folder and racingroach writes a dated bundle of four dashboard-ready spreadsheets — an overall summary, content types, status codes by depth, and issues by kind — plus a README that walks you through connecting them and shows which chart to build from which file. No Google account, no upload, no API: it's a plain folder of files you connect yourself. Each export gets its own dated folder, so snapshots stack up over time instead of overwriting each other.
The interface now speaks nine languages
Pick your language in Settings → General and the whole app switches instantly, no restart: menus, buttons, tables, settings, and the explanation and fix for every one of the 100 SEO checks. English, German, Spanish, French, Portuguese (Brazil), Italian, Dutch, Polish and Japanese, all free. The existing language setting is now labelled "Target site language", since that one tells racingroach what language the site you're crawling is in.
Caching and security headers on every page
Open any crawled page and its overview now shows the Cache-Control and Content-Encoding headers the server sent, plus which security headers were present — HSTS, CSP, X-Content-Type-Options and X-Frame-Options. racingroach was already collecting all of this during the crawl; now you can see it.
Screenshots, JavaScript errors, and page archives (premium)
When you crawl with JavaScript rendering, racingroach now keeps what the browser saw: the JavaScript errors each page threw, and — if you turn them on — a full-page screenshot and a copy of the rendered page. Open any page to view its screenshot, read its console errors, or reopen the archived page offline, and export the whole set as a single zip.
Excel export, and an all-in-one workbook (premium)
Every export that offered CSV now also saves as Excel — just pick .xlsx in the save dialog and you get a formatted sheet with a frozen header and filters ready to go. Premium adds a single workbook for the whole audit: an overview tab, your pages and external links, one tab per issue found, and the structured-data, spelling and elements tables.
Scheduled crawls (premium)
Save a crawl as a reusable profile and have it run on a schedule — hourly, daily, weekly, monthly, or once — while the app is open, with each run written to its own dated file and its result waiting for you on the new Schedules screen.
In-app update check
The About panel now checks for updates on demand and tells you when a new version is staged and ready to apply the next time you quit.
JavaScript rendering is free, for good
Rendering pages in a real headless browser stays in the free tier permanently — now written into the spec rather than left as an open product call.
Custom robots.txt override (premium)
Test a robots.txt change against a real crawl before you deploy it: supply your own rules per host and see exactly what gets fetched and flagged.
Link Score
A PageRank-style 0–100 score for every page, turning your internal link graph into one sortable measure of internal authority.
Bulk link exports
Get the whole internal link graph back out — every link with its anchor text and follow status — in a filterable Links screen and a streaming CSV export.
SERP snippet preview
See how each page's title and meta description will render in Google, with pixel-width truncation warnings and editable rewrites you can export.
Crawl visualisations
Force-directed crawl diagrams and hierarchical trees for spotting depth problems, orphaned clusters, and structural imbalance at a glance.
Crawl configuration options
Additional crawl limits, directive-following (canonical, next/prev, HSTS, meta refresh), URL rewriting, CDN-as-internal scoping, and resource-crawl toggles — all free.
Page detail views & custom extraction
Click any row to inspect one URL in detail, with its inlinks and outlinks. Custom extraction (premium) pulls any XPath, CSS, or regex match off every page into its own column.
Compare crawls (premium)
Answer "what changed since the last crawl?" — diff two crawls of the same site after a migration, a release, or on a monitoring cadence.
Headless CLI mode
Run a crawl, export results, and check issue counts from the command line — no UI, so crawls fit straight into CI and build pipelines.
Segments
Define custom URL segments and get a per-segment breakdown of page counts, status codes, and issue kinds.
Custom JS snippets (premium)
Run your own JavaScript against every crawled page to extract custom per-page data.
Accessibility auditing
Automated accessibility checks now run alongside the rest of the technical-SEO audit.
Custom source search
Search rendered page content for exact strings or patterns across an entire crawl.
Spelling & grammar checks
Flags likely spelling and grammar issues in page content during the crawl.
Crawl-overview dashboard
A live summary screen — pages by status, top issue kinds, and progress at a glance.
XML sitemap generation & list mode
Generate a size-split XML sitemap from a crawl, or crawl a fixed list of URLs instead of discovering them.
Auto-update
The desktop app now checks for and applies signed updates in the background.
Licensing & free tier
Introduced the free tier (1,000 pages/crawl) and the offline-verified annual license.
Elements view
A dedicated screen for per-page element data — headings and the full image inventory.
Earlier releases added the core crawl engine, the desktop shell, and the first 80+ SEO checks.
Try the latest build.
Every release ships to the free tier first — no separate beta channel.