Noindex in Sitemap
SkillDev toolsnoindex-in-sitemap is an agent skill for auditing an XML sitemap and diagnosing indexing issues. It guides the agent through checking whether URLs listed in a sitemap also carry a noindex meta tag or X-Robots-Tag header, then explains how to resolve the contradiction. It applies to any site where such directives may conflict with sitemap entries.
Available today. Use it from your connected AI after setup.
No other account needed.
Have the skill file installed so the agent can load it when auditing a sitemap.
Then ask your AI: use the Noindex in Sitemap skill
What your AI can do with it
- Fetch an XML sitemap and check each listed URL for noindex directives
- Detect <meta name='robots' content='noindex'> tags in page heads
- Detect X-Robots-Tag: noindex HTTP response headers
- Report sitemap URLs that also carry a noindex directive
- Check for sitemap entries returning non-200 status codes
- Recommend whether to remove the noindex directive or the sitemap URL
Getting started
- Have the skill file installed so the agent can load it when auditing a sitemap.
- Have the URL of the XML sitemap or sitemaps to audit.
- Ask the agent to check each sitemap URL for noindex meta tags or X-Robots-Tag headers.
- Review the reported URLs and decide for each whether the page should be indexed.
- Apply the fix: remove the noindex directive or remove the URL from the sitemap, never both.
What this skill tells your AI
The instructions your AI receives, as published by thedaviddias/front-end-checklist in skills/noindex-in-sitemap/SKILL.md and read by ahel’s review.
Listing noindexed pages in your sitemap sends contradictory signals to Googlebot—it wastes crawl budget and may confuse Google into ignoring the noindex directive or spending crawl time on pages you don't want indexed.
Quick Reference
- Never include noindexed pages in your XML sitemap
- The sitemap and noindex directive send contradictory signals to crawlers
- Sitemaps should only list canonical-url, indexable, 200-status URLs
- Remove noindexed URLs from sitemap or remove the noindex directive—pick one
Check
Fetch the XML sitemap(s) and check each listed URL. For each URL, retrieve the page and check for in the or an X-Robots-Tag: noindex HTTP header. Report any URLs present in the sitemap that also carry a noindex directive.
Fix
For each URL that has a noindex directive AND appears in the sitemap: decide whether the page should be indexed. If yes — remove the noindex directive. If no — remove the URL from the sitemap. Never leave both in place.
Explain
The XML sitemap is a recommendation to search engines: 'please crawl and index these pages'. A noindex directive is an instruction: 'do not index this page'. Including noindexed pages in the sitemap creates a contradiction—Google will resolve it by following noindex, but the URL still gets crawled, wasting crawl budget.
Code Review
Fetch each URL in the XML sitemap. For each URL, check the HTTP response for X-Robots-Tag: noindex header, and check the rendered for . Report any URL that appears in the sitemap AND carries a noindex directive. Also check for sitemap entries returning non-200 status codes.
For full implementation details, code examples, and framework-specific guidance,
see references/rule.md.
Rule page: https://frontendchecklist.io/en/rules/seo/noindex-in-sitemap
Signals
- GitHub stars
- 74k
- Forks
- 7k
- Last commit
- Aug 2026
Questions
- Why is it a problem to list noindexed pages in a sitemap?
- The sitemap asks crawlers to index the pages while noindex says not to. Google follows noindex but still crawls the URL, wasting crawl budget and sending contradictory signals.
- How is the check performed?
- Fetch each URL in the XML sitemap, check the HTTP response for an X-Robots-Tag: noindex header, and check the rendered head for a noindex meta robots tag. Report any URL in both states.
Advanced
- Item type
- skill
- Key
noindex-in-sitemap- Source
- github.com/thedaviddias/front-end-checklist