Sitemaps, internal links and crawl budget: what small sites really need

'Crawl budget' gets a lot of attention, but for most small and medium sites it is not the real problem. Google can crawl a few thousand pages without strain. What matters at that size is making your pages easy to find and worth keeping.

What a sitemap does

An XML sitemap is a list of the URLs you want crawled. It helps discovery, especially for new pages and pages with few links, but it is a hint, not a command. Keep it accurate.

• List only canonical URLs that return 200 and can be indexed.
• Leave out redirects, noindex pages, parameter duplicates and pages blocked by robots.txt.
• Update it when you publish, and submit it once in Search Console.

Why internal links matter more

Crawlers move through your site by following links. A page with no internal links is hard to find and tells Google it is not important. Link new pages from related older ones, and link your most valuable pages from your homepage or main navigation.

When crawl budget does matter

• Very large sites with hundreds of thousands of URLs.
• Sites that generate endless URL variations through filters, sessions or calendars.
• Sites with slow servers or frequent errors, which cause Google to back off.

A simple routine

1. Publish the page with at least one internal link from an indexed page.
2. Add it to the sitemap.
3. Request indexing for important pages only.
4. Check the Pages report after a week or two and fix what it reports.

Common sitemap mistakes

• Including URLs that redirect, return errors or carry a noindex tag.
• Listing every parameter variation of the same page instead of the one canonical URL.
• Leaving old, deleted URLs in the file for months.
• Never telling Search Console where the sitemap is, so you never see its errors.
• Using relative or inconsistent URLs (http in the sitemap, https on the site).

Internal linking patterns that work

• Hub pages: one strong page per topic that links to every related post, and that every related post links back to.
• Contextual links: link from inside the body text with descriptive anchor text, not 'click here'.
• Related-post blocks and breadcrumbs, which give every page several ways in.
• No orphans: every page you want indexed should be reachable by a normal link within a few clicks of the homepage.

Monitoring in Search Console

The Sitemaps report shows how many URLs you submitted and how many Google found valid. A large gap is a signal to look at the Pages report and read the reasons. The Crawl stats report shows how often Google fetches your pages and how your server responds. Slow or error-prone responses are the quickest way to lose crawl attention.

A five-minute audit

1. Open your sitemap and click ten random URLs. Each should load normally and be the page you want indexed.
2. Pick three important pages and check how many internal links point to them.
3. Look at the Pages report for the top two reasons URLs are not indexed, and fix those first.