Noindex In Google Search Console

Google’s John Mueller recently answered a question about phantom noindex errors reported in Google Search Console. Mueller asserted that these reports may be real.

---

What is a noindex directive?

A noindex robots directive is one of the few commands that Google must obey, one of the few ways that a site owner can exercise control over Googlebot, Google’s indexer.

Yet it’s not totally uncommon for Search Console to report being unable to index a page because of a noindex directive that seemingly does not exist in the HTML code.

When Google Search Console (GSC) reports “Submitted URL marked ‘noindex’,” it is reporting a seemingly contradictory situation:

• The site asked Google to index the page via an entry in a Sitemap.

• The page sent Google a signal not to index it (via a noindex directive).

It’s a confusing message from Search Console that a page is preventing Google from indexing it when that’s not something the publisher or SEO can observe at the code level.

The person asking the question posted on Bluesky:

“For the past 4 months, the website has been experiencing a noindex error (in ‘robots’ meta tag) that refuses to disappear from Search Console. There is no noindex anywhere on the website nor robots.txt. We’ve already looked into this… What could be causing this error?”

---

Noindex shows only for Google

Google’s John Mueller answered the question, sharing that there were always a noindex showing to Google on the pages he’s examined where this kind of thing was happening.

Mueller responded:

“The cases I’ve seen in the past were where there was actually a noindex, just sometimes only shown to Google (which can still be very hard to debug). That said, feel free to DM me some example URLs.”

While Mueller didn’t elaborate on what can be going on, there are ways to troubleshoot this issue to find out what’s happening.

---

How to troubleshoot phantom noindex errors

It’s possible that some code is causing a noindex to appear only for Google. For example, a page may have once had a noindex and a server‑side cache (e.g., a caching plugin) or a CDN (e.g., Cloudflare) cached the HTTP headers from that time. The old noindex header could then be shown to Googlebot (which frequently visits the site) while serving a fresh version to a regular visitor.

Check the HTTP header – many free tools exist, such as:

• KeyCDN HTTP Header Checker

• SecurityHeaders.com

A 520 server response code is sent by Cloudflare when it’s blocking a user agent.

Screenshot: 520 Cloudflare Response Code

Image: “Screenshot showing a 520 error response code”

https://cdn.searchenginejournal.com/wp-content/uploads/2026/01/520-response-cloudflare-555.png

Screenshot: 200 Server Response Code

Image: “Raw Headers contain Response code, HTTP/2, date, content‑type, server, vary, expires, and cache‑control”

https://cdn.searchenginejournal.com/wp-content/uploads/2026/01/header-response-code-601.png

I checked the same URL with two different header checkers; one returned a 520 (blocked) response, the other a 200 (OK) response. This shows how Cloudflare can respond differently depending on the requester. Try several checkers to see if a consistent 520 response appears.

---

Using Google’s Rich Results Test

When a page shows something exclusively to Google that isn’t visible in the source code, you can force Google to fetch the page using an actual Google crawler (from a Google IP address) via the Rich Results Test.

• The request originates from Google’s data centers (Google IP).

• It passes reverse‑DNS checks, so any IP‑based blocking will be revealed.

If the page is blocked by a noindex, the tool will be unable to provide structured‑data results and will display a status such as “Page not eligible” or “Crawl failed.” Expanding the error details should show something like “Robots meta tag: noindex.”

The Rich Results Test uses the Google‑InspectionTool/1.0 user‑agent, not the generic Googlebot UA, so IP‑based blocks are still caught.

---

Spoofing the Googlebot user‑agent

If a rogue noindex tag is specifically written to block Googlebot, you can mimic the Googlebot UA:

• Chrome User Agent Switcher extension – Chrome Web Store link

• Configure a tool like Screaming Frog to identify itself as Googlebot.

Screenshot: Chrome User Agent Switcher

Image: “User‑Agent Switcher for Chrome is a Google‑featured extension with a 3.9‑star rating from 2.7K users and 1,000,000 users overall”

https://cdn.searchenginejournal.com/wp-content/uploads/2026/01/chrome-user-agent-switcher-560.png

---

Phantom noindex errors in Search Console

These errors can be painful to diagnose, but before giving up, follow the steps above to uncover the hidden cause:

1. Verify HTTP headers for stray noindex directives.

2. Test the URL with multiple header checkers.

3. Run the URL through Google’s Rich Results Test to see what Google actually receives.

4. If needed, spoof the Googlebot user‑agent to catch UA‑specific blocks.

---

Featured Image by Shutterstock/AYO Production