What It Actually Means
It is not infinite and it is not equal across sites. It scales roughly with how important the site appears to be, which in practice means how much external evidence exists that it matters.
Why It Is Usually Ignored, And When It Is Not
For most small sites this is genuinely not a concern. A fifty-page site gets crawled comfortably and nobody needs to think about it.
It becomes the dominant constraint in one specific situation, and it is a situation businesses walk into deliberately: a site with modest authority that has published a large number of pages.
That combination produces a familiar and confusing outcome. Pages are published, submitted, technically sound, and simply sit there. Search Console reports them as discovered but not indexed, or crawled but not indexed, and no amount of on-page work changes it. Nothing is broken. The site is being rationed.
Why More Pages Can Make It Worse
This is the counterintuitive part, and it is the reason the usual response fails.
The instinct when a site is not getting traffic is to publish more. On a rationed site, every additional page divides the same limited attention further. New pages take longer to be discovered, existing pages get revisited less often, and updates take longer to register.
So a site publishing steadily can find that its indexing rate falls while its page count rises, and the more effort goes in the worse the symptom looks.
The way out is not more pages. It is fewer, better ones, and more evidence that the site deserves attention.
What Actually Consumes It
Thin and near-duplicate pages. Several pages covering the same intent consume budget and compete with each other. Consolidating them frees attention and removes the competition at the same time, which is why it helps twice.
Redirect chains. Each hop is a fetch. Redirects pointing at redirects waste budget invisibly.
Parameter and filter URLs. Faceted navigation can generate thousands of near-identical addresses. On an e-commerce site this is frequently the whole problem.
Broken links and error pages. Crawling something that returns nothing is spent budget.
Slow responses. Crawlers back off from slow servers, which directly reduces how much gets fetched.
What Actually Increases It
Honestly: authority. Links from credible sites are what persuade a search engine a site is worth more attention, and there is no technical substitute.
Everything else is efficiency. A clean sitemap listing only pages worth indexing, sensible internal linking so important pages are easy to reach, fast responses, no chains, and no thin duplicates. Those make better use of what you have, and the ceiling is set elsewhere.
What To Ask
- How many pages do we have, and how many are indexed? The gap is the diagnosis.
- How many pages are worth indexing? Frequently far fewer than exist.
- Where are the duplicates? Same intent served by several pages is the most common cause and the most fixable.
- What is in the sitemap? It should contain only pages you would be pleased to have found.
- How many referring domains do we have? If the answer is close to none, that is the constraint, and technical work will not move it.
The Uncomfortable Conclusion
For a site in this position, the productive work is usually reduction and consolidation rather than production, plus a genuine effort to earn a small number of good links.
That is a harder sell than a content calendar, and it is the version that works. Publishing into a site that is not being crawled is effort that arrives nowhere, and it feels like progress the entire time.
More terms are in the glossary.
