Indexing & snippet eligibility
Last updated: 2026-11-16
Googlebot can crawl and index the site #
Google requirement
What it is #
Two things stop Google from indexing a page:
- robots.txt blocking
Googlebot; - a
noindexdirective (<meta name="robots" content="noindex">or anX-Robots-Tag: noindexheader).
Why it matters #
A page that can't be crawled or indexed can't appear in Google Search or in its AI features (AI Overviews, AI Mode), which draw on the Search index. This is the most common "why am I invisible?" cause after a redesign: a noindex left over from a staging site.
How we check it #
We parse your robots.txt for rules that block Googlebot from /, and read your homepage for robots or googlebot meta tags containing noindex or none.
How to fix #
- robots.txt: remove any
Disallow: /that applies toGooglebotor to*. - Meta tag: delete
noindexfrom<meta name="robots">(check your framework's SEO plugin or page settings, and staging-only environment variables that leaked to production). - Header: check your CDN or host for an
X-Robots-Tagheader.
Pages can show a snippet #
Google requirement
What it is #
A snippet is the text Google shows under your link. nosnippet and max-snippet:0 tell Google not to show one.
Why it matters #
Google's AI features only use pages that are eligible to show a snippet. Opting out of snippets quietly opts you out of AI Overviews and AI Mode as well.
How we check it #
We read your homepage's robots meta tags for nosnippet or max-snippet:0.
How to fix #
Remove nosnippet and max-snippet:0 unless you truly want no preview. To hide one passage (a price table, a legal notice), wrap that element in data-nosnippet instead of opting out the whole page.
Also check: AI features inclusion (manual) #
Search Console → Settings has a control for including your site in Search's generative AI features. We can't read it for you. Open it once and confirm your site isn't excluded.
Verify it yourself #
curl -s https://yourdomain.com/ | grep -io '<meta[^>]*robots[^>]*>'
curl -s https://yourdomain.com/robots.txtThen check the live page in Search Console → URL Inspection → Test live URL.
Tips #
- "Discovered – currently not indexed" isn't a directive problem. It usually means Google hasn't prioritized the page yet. Internal links, a sitemap and genuinely useful content help. So does patience.
- Noindex needs crawling to work. If robots.txt blocks a page, Google can't see its
noindex, so the URL can still appear in results. Allow crawling if you want thenoindexrespected.
Sources #
- Google: Robots meta tag, data-nosnippet and X-Robots-Tag
- Google: Snippets
- Google: AI features and your website
- Search Console Help: Search generative AI control
- Search Console Help: URL Inspection tool