Free guides

Crawl budget & technical SEO guides

Short, practical explainers on how search engines crawl your site, how JavaScript rendering and Core Web Vitals affect what gets seen, and how to manage AI crawlers in robots.txt. Each guide links to the Google Search Central (and related) documentation it's based on. Want a quick read on your own site first? Try the free Crawl Budget Snapshot.

Crawl budget explained
Crawl capacity vs. crawl demand, which sites actually need to worry, and the fixes Google recommends.
How to find crawl traps
Spot the URL patterns that generate near-infinite URLs and waste crawling, then close them off safely.
Reading the Google Search Console Crawl Stats report
What each chart and breakdown means, and which patterns deserve a closer look.
robots.txt checklist
Location, syntax, rule precedence, HTTP status handling, and the mistakes that bite most often.
XML sitemap best practices
Size limits, index files, which URLs belong, how lastmod is used, and how to submit.
JavaScript SEO and rendering
Crawl → render → index, soft 404s in SPAs, History API vs fragments, and common JavaScript SEO mistakes.
AI crawlers and robots.txt
Vendor tokens for training vs search/retrieval, how Allow/Disallow works, and a free checker for your robots.txt.
Core Web Vitals explained
LCP, INP and CLS definitions and thresholds, field vs lab measurement, and practical optimization directions from Google.