What It Actually Means
It is a hint rather than an instruction. Listing a page does not guarantee it will be indexed, and omitting one does not prevent it. What it does is help discovery, particularly for pages that are not well linked from elsewhere on the site.
What It Is Actually For
Discovery, and to a lesser extent, signalling freshness.
Search engines find most pages by following links. A page nobody links to is hard to discover, and a sitemap is how it gets found. That makes sitemaps most valuable on large sites, new sites with few external links, and sites where some content is buried deep in the structure.
The last-modified date matters more than people realise. Accurate dates help a crawler prioritise what to revisit. Dates that update every time anything on the site changes are worse than no dates, because they tell the crawler everything changed, which means nothing did.
The Rule That Matters
A sitemap should contain only pages you would be pleased to see in search results.
This is where most sitemaps go wrong, because they are generated automatically and include everything: thin pages, tag archives, paginated lists, thank-you pages, noindexed pages, redirects, and things that were never meant to be found.
That causes a specific problem on a site already short of crawl budget. The sitemap is an invitation, and inviting a crawler to spend attention on hundreds of pages not worth indexing consumes exactly the attention the pages that matter need.
It also degrades the signal. A sitemap where a large share of listed pages are never indexed tells the search engine your judgement about your own site is unreliable.
What Should Not Be In It
Noindexed pages, which is a direct contradiction. Redirected addresses. Anything returning an error. Canonicalised duplicates, since only the canonical belongs. Thin archive and tag pages, unless they genuinely serve a purpose. Thank-you and confirmation pages.
Auditing this is quick and it is rarely done, because the file is generated by a plugin and assumed to be correct.
What To Ask
- How many pages are in our sitemap, and how many are indexed? A large gap is the diagnosis.
- Does it include anything noindexed, redirected or broken? Frequently yes.
- Are the last-modified dates accurate? Or do they all update together.
- Is it referenced in robots.txt and submitted in Search Console? Both are trivial and both get missed.
- Do the addresses match our canonicals exactly? Including protocol, www and trailing slash.
The Practical Version
For most sites this is a small job done once and reviewed occasionally: generate it, remove what does not belong, keep it accurate, submit it.
Its value is not that it gets pages indexed. It does not. Its value is that it makes your intentions unambiguous, and on a site where attention is limited, being clear about which pages actually matter is worth more than listing everything you have.
More terms are in the glossary.
