A technical SEO audit checklist for a live site

// In this article+

A technical SEO audit checklist for a live site should run in dependency order. First, can Google reach the page. Second, can it index the page. Third, which URL does it pick as the canonical. Only then do rendering and speed matter, because a page that fails the first three steps never gets to use the rest.

Run the first three in the first hour. They find the problems that cost the most traffic, and each one has a named Google report behind it.

What belongs on a technical SEO audit checklist for a live site?

NUOPTIMA defines a technical SEO audit as "a systematic review of how search engines crawl, render, and index a website, identifying the technical issues that prevent it from ranking and the areas where it can grow." That definition is sound. The trouble is the list that follows it: NUOPTIMA's checklist ↗︎ runs 13 sections, from internal pages and redirects to social tags and CSS, and it does not rank them by impact.

A flat list suits a new site with nothing to lose. A live site already earns traffic, so the job is different. You are looking for the one thing that is quietly removing pages from Google, and you want to find it before you read about image formats.

Test eligibility first. Google's technical requirements ↗︎ say "Google only indexes pages that are served with an HTTP 200 (success) status code." Everything else in the audit assumes that is true for your pages.

Then follow the dependency. Carril Agency's audit checklist ↗︎ makes the same call on priority: "If Google cannot find or index your pages, nothing else matters." I would add the reason. Each later check reads a page that an earlier check had to let through.

The order, with the Google report that answers each step:

1. Reach. Status codes, redirects, robots.txt. Tool: URL Inspection and a status check on top landing pages.
2. Index. noindex conflicts and index status. Tool: the page indexing report.
3. Canonical choice. Which URL Google selected. Tool: URL Inspection, the Google-selected canonical field.
4. Render. What Google sees after JavaScript runs. Tool: URL Inspection, live test.
5. Speed and crawl load. Response time and Core Web Vitals. Tool: Crawl Stats and the Core Web Vitals report.

The canonical step has its own rules; the post on canonical tags and when to use one covers them. For what the whole discipline covers beyond an audit, see Technical SEO: what the work actually covers.

What do you check in the first hour?

Three checks, in order: status and redirects, robots.txt against noindex, then the page indexing report. Use your 20 to 50 highest-traffic landing pages from Search Console as the sample, not a full crawl.

Source: Google Search technical requirements ↗︎ for the 200 status rule.

Status codes and redirects

Google's documentation on HTTP status codes ↗︎ sets the limit for chains: "By default, Google's crawlers follow up to 10 redirect hops." Ten is a ceiling, not a target. Anything past one or two hops is a cleanup item, and it usually points to a migration that stacked redirects on redirects.

The same page covers 4xx, 5xx and soft 404 handling. A soft 404 is a page that shows an error message to visitors but returns a 200 code, so the status check alone will not catch it. Look for thin "not found" templates on your top URLs.

robots.txt versus noindex

These two controls do different jobs, and audits confuse them constantly. Google's robots.txt introduction ↗︎ says a robots.txt file "is not a mechanism for keeping a web page out of Google." It controls crawling, not indexing.

The conflict is the one to hunt for. Google's noindex guide ↗︎ says that for noindex to work, "the page or resource must not be blocked by a robots.txt file, and it has to be otherwise accessible to the crawler". A page with both a noindex tag and a robots.txt block can stay in the index, because Google never reads the tag.

Check: for each URL you meant to remove, confirm it is not also disallowed in robots.txt.

The page indexing report

Open the page indexing report ↗︎ in Search Console and group the URLs by reason. Read the statuses literally. For Crawled - currently not indexed, Google's definition is "The page was crawled by Google but not indexed. It may or may not be indexed in the future."

That wording is a hedge, and you should treat it as one. Do not assume a fix will move the page. Sort the not-indexed groups by whether a page in them earns or should earn traffic, and investigate those first.

How do you check what Google actually renders and can crawl?

Crawlability is whether Googlebot can find and fetch a page. Rendering is whether Google can see the content after the page's JavaScript has run. Test both on one URL from each key template: home, category, product or service, and article.

Compare raw and rendered HTML. Google's guide to JavaScript SEO basics ↗︎ tells you to "use the Rich Results Test or the URL Inspection Tool and look at the rendered HTML". Put the raw source next to the rendered version. If the main text, the title, the canonical or the internal links exist only in the rendered version, the page depends on rendering to be understood.

Check that internal links are real links. Google's link best practices ↗︎ say Google can only crawl a link that is an <a> element with an href attribute. Click handlers on a div or a button that navigates with script do not count. A page reachable only through such elements is effectively an orphan, even if visitors find it.

Use the indexed view and the live view for different questions. The URL Inspection tool ↗︎ states: "The results shown are from most recently indexed version of a page, not the live version on the web." The indexed view tells you what Google holds today. The live test tells you whether a fix you shipped works.

The three tests, in the order to run them:

1. Raw HTML against rendered HTML on three templates.
2. Anchor-with-href check on the main navigation and on one in-content link per template.
3. Indexed view against live test on the one URL where something looks wrong.

A worked example: a service page appears in the page indexing report as not indexed. The URL Inspection indexed view shows an empty body. The live test shows full content. Google last saw the page before the fix, so the action is to request indexing, not to change the template again.

When is the Crawl Stats report worth opening?

For most sites, later than you would expect. The Crawl Stats report ↗︎ shows host status, response codes and average response time over recent crawling. Google's own help page is direct about who needs it: "This report is aimed at advanced users. If you have a site with fewer than a thousand pages, you should not need to use this report."

Skip it on a small site and spend the time on the indexing and rendering steps. On a larger site, open it with a specific question:

• Host status. Did Google hit robots.txt fetch failures, DNS errors or server connectivity problems recently?
• Response codes. Is the share of 5xx or 404 responses climbing?
• Response time. Did average response time rise after a release or a hosting change?

A rising response time with a falling crawl count is the pattern to chase. If the cause is page speed on the front end rather than the server, that belongs to Core Web Vitals optimization, which has its own audit.

How do you rank what you found?

Rank by traffic at stake, not by how bad the issue looks. A noindex tag on one low-value page is a smaller finding than a slow template that serves half your organic landing pages.

A triage scheme that works on a live site:

• Blocker. Pages that earn traffic are blocked, non-200, noindexed or unrendered. Fix first.
• Leak. Redirect chains, wrong canonical choice, soft 404s on pages with links or traffic. Fix this sprint.
• Debt. Issues on pages with no traffic and no links. Schedule, do not rush.

Keep a findings log with five columns: URL or template, issue, report that showed it, monthly organic clicks at stake, and owner. The clicks column decides the order.

Benchmarks help you tell a real problem from background noise. The HTTP Archive's Web Almanac SEO chapter ↗︎ found that 85% of requests for robots.txt files returned a valid 200 status code in 2025, and that the 404 rate for robots.txt fell to 13%. A 404 on robots.txt means unrestricted crawling, so it is common and harmless. A 5xx is not. The same chapter says a 5xx response on robots.txt "can cause search engines to temporarily treat the site as blocked".

The chapter also found invalid elements in the <head> on 10.1% of desktop pages and 10.3% of mobile pages. That matters because a canonical tag or robots directive that sits below an invalid element can end up in the body, where it has no effect. If your canonical choice looks wrong in URL Inspection, view the rendered head before you rewrite a rule.

On cadence, Carril's guide says to run the audit quarterly, and after any major site update, redesign or migration. Treat that as a floor. A site that ships weekly should run the first-hour checks after each release, and the SEO migration checklist for launch week covers the extra checks a relaunch needs.

What to do this week

1. Export the page indexing report and group the not-indexed URLs by reason. Mark the groups that contain pages that earn traffic.
2. Check the top landing pages for status and conflicts. Confirm each returns 200, is not disallowed in robots.txt, and carries no noindex it should not have.
3. Compare raw and rendered HTML on three templates. Note anything that only appears after JavaScript runs.
4. Check that internal links use anchors with an href. Test the main navigation and one in-content link per template.
5. Log findings by traffic at stake and put a re-run on the calendar, quarterly at the latest.

If you want the same work done and documented for you, that is what SEO audit services cover. The checklist above is enough to find the expensive problems on your own.

FAQ

How long does a technical SEO audit take?

Carril Agency's guide gives 2 to 4 hours of analysis for a site under 500 pages, plus time to document findings, and 1 to 3 days for larger sites. That is one agency's estimate, not a standard. The first-hour checks in this checklist are a subset you can finish much faster on a sample of top pages.

Does a robots.txt block remove a page from Google?

No. Google's robots.txt introduction says the file is not a mechanism for keeping a page out of Google. To keep a page out of the index, use noindex, and leave the page crawlable so Google can read the tag, as Google's noindex documentation explains.

How often should I re-run the audit?

Carril's guide recommends quarterly for most sites, and again after a redesign, a migration, a major update or a sharp traffic drop. Sites that release often should rerun the first-hour checks after each release.