Indexing

Getting thousands of URLs crawled and indexed: sitemaps, canonicals, crawl budget, and pages Google decides not to keep.

Programmatic SEO / Indexing

Programmatic SEO AI agent rules: letting Grok crawl safely

Which Grok user-agent tokens to name in robots.txt, how to open entity pages but keep facets closed, and how to rate-limit agents without hurting Googlebot.

Maciej Wiatr

7 min read

Programmatic SEO deindexing recovery: prune, merge or rebuild

How to diagnose crawled-but-not-indexed programmatic pages by template, decide what to prune, merge or improve, and measure recovery per template.

Maciej Wiatr

7 min read

Programmatic SEO in Next.js: ISR, 404s and sitemaps at scale

How to build 50,000+ programmatic pages in Next.js 15 and 16: what to prerender, ISR for the long tail, real 404 status codes, and sharded sitemaps Google trusts.

Maciej Wiatr

9 min read