Programmatic SEO in Next.js: ISR, 404s and sitemaps at scale

How to build 50,000+ programmatic pages in Next.js 15 and 16: what to prerender, ISR for the long tail, real 404 status codes, and sharded sitemaps Google trusts.

Maciej Wiatr

9 min read

Read as markdown

Programmatic SEO in Next.js comes down to four decisions on one dynamic route: which pages you build ahead of time, which you render on first visit, which return a real 404, and how the sitemap tells Google what changed. The App Router handles all four, but the APIs differ between Next.js 15 and Next.js 16 with Cache Components switched on, and most tutorials ignore that. This guide covers both setups, at the scale where it matters: 50,000 pages and up.

Versions matter here. Several of the APIs below were added in Next.js 16.3 and 16.4, and the docs pages were updated in September and October 2026. Check your version before you copy anything.

How do you set up programmatic SEO on the Next.js App Router?

The basic layout is the same in 15 and 16:

The dataset does most of the work. If a row has nothing on it that the next row doesn’t also have, rendering won’t fix it. The data hub covers that side.

How to implement programmatic SEO in Next.js 15 (or 16 without Cache Components)

Without Cache Components you still have the route segment options most tutorials use: revalidate and dynamicParams.

// app/[category]/[slug]/page.tsx
import type { Metadata } from "next";
import { notFound } from "next/navigation";
import { getEntry, getTopEntries } from "@/lib/data";

type Props = { params: Promise<{ category: string; slug: string }> };

export const revalidate = 3600;     // ISR: regenerate at most hourly
export const dynamicParams = true;  // unlisted slugs render on first visit

export async function generateStaticParams() {
  const top = await getTopEntries(5000);
  return top.map((e) => ({ category: e.category, slug: e.slug }));
}

export async function generateMetadata({ params }: Props): Promise<Metadata> {
  const { category, slug } = await params;
  const entry = await getEntry(category, slug);
  if (!entry) notFound();
  return {
    title: `${entry.name} in ${entry.cityName}`,
    alternates: { canonical: `/${category}/${slug}` },
  };
}

export default async function Page({ params }: Props) {
  const { category, slug } = await params;
  const entry = await getEntry(category, slug);
  if (!entry || entry.listings.length === 0) notFound();
  return <EntryTemplate entry={entry} />;
}

A few details from the generateStaticParams docs:

params is a Promise. That has been the case since Next.js 15, and the Next.js 16 release post makes awaiting it mandatory. Older guides that read params.slug directly will break.

How to implement it in Next.js 16 with Cache Components

With cacheComponents: true, the route segment config docs say dynamic, dynamicParams, revalidate and fetchCache are removed as of v16.0.0. Caching moves into the data function:

import { cacheLife, cacheTag } from "next/cache";

async function getEntryCached(category: string, slug: string) {
  "use cache";
  cacheLife("max");
  cacheTag(`entry:${category}:${slug}`);
  return getEntry(category, slug);
}

Three more changes apply:

For a fully on-demand route, the docs suggest a placeholder such as [{ slug: '__placeholder__' }] that you handle with notFound(). They also warn that it “prevents build time validation from working effectively and may cause runtime errors.” I’d rather return one real, popular slug.

Should I use generateStaticParams for all 50,000 pages or ISR?

Use ISR for most of them. The ISR with Cache Components guide says: “Every page you prerender increases build work and produces output that has to be stored and deployed. Many routes may never be visited before your next deployment.” Its advice is to prerender “popular pages or predictable content.”

The Smashing Magazine ISR guide says build time scales linearly with page count. A rough estimate on DEV puts 100,000 pages at about 50ms each at “almost 2 hours”. That is arithmetic, not a benchmark.

One case study shows the split in practice. A jobs site with about 180k pages, as reported by 72Technologies, made three changes:

Builds went from 2h40m to 11 minutes, and listings went from up to 24 hours stale to about 10 minutes. The numbers are self-reported, with no raw data and no traffic figures.

ChoiceBuild timeFirst visitFreshness
Prerender everythingGrows with every pageFastOnly as fresh as the last deploy
Prerender head, ISR the tailSmallSlower for unlisted pagesSet by revalidation
Fully on demandMinimalSlowest for first visitorSet by revalidation

If you self-host, the ISR docs say the cache lives on the server’s filesystem by default, and ISR only works on the Node.js runtime.

What does dynamicParams = false do, and how do I return a 404 for pages with no data?

The dynamicParams docs explain the switch:

false only works if you prerender every valid page, which is the opposite of the advice above. For ISR sites the useful tool is notFound(). It also covers rows that exist but are empty, like a city with zero listings.

The streaming 200 trap

The loading.js docs say “Streaming page responses begin with a 200 status code.” If notFound() runs after streaming has started, either inside <Suspense> or below a loading.tsx, the response stays 200 with a noindex meta tag. Next.js admits “Some crawlers may label these responses as ‘soft 404s’.” Google’s crawl budget guide says “soft 404 pages will continue to be crawled, and waste your budget.”

Partial Prefetching makes this worse. From Next.js 16.3, unlisted params get a generic App Shell instantly. Any 404 decided after that is already too late for the status code. Neither doc says this in one place; it’s my reading of the two together.

To get a real 404:

  1. Await params, load the row and call notFound() at the top of the page, outside any Suspense boundary. Don’t put a loading.tsx above the route.
  2. On 16.4+ with Cache Components, set ensureStatic = 'navigation'. The ensureStatic docs say this makes the first request wait for the full page instead of a shell. It fails the build on uncached data, cookies(), headers(), server-side searchParams, or short cache lifetimes.
  3. Check slug existence in proxy.ts (the renamed middleware.ts), as long as the check is fast.

The notFound docs list two smaller traps. A try/catch around it swallows the 404 unless you use unstable_rethrow. Calling it inside an un-awaited promise renders no not-found UI.

Curl a known-empty slug after every deploy and check the status line.

How do I split a Next.js sitemap past 50,000 URLs with generateSitemaps?

Google’s sitemap docs cap each file at “50MB (uncompressed) or 50,000 URLs” and ignore <priority> and <changefreq>. generateSitemaps shards the output. Since v16.0.0, id is a Promise<string>.

The official example has four bugs to avoid:

This version fixes all four:

// app/places/sitemap.ts
import type { MetadataRoute } from "next";
import { countEntries, getEntriesPage } from "@/lib/data";

const PER_FILE = 45000; // headroom under 50,000

export async function generateSitemaps() {
  const total = await countEntries({ indexable: true });
  const files = Math.ceil(total / PER_FILE);
  return Array.from({ length: files }, (_, i) => ({ id: i }));
}

export default async function sitemap(props: {
  id: Promise<string>;
}): Promise<MetadataRoute.Sitemap> {
  const id = Number(await props.id);
  // ORDER BY a stable key, LIMIT PER_FILE OFFSET id * PER_FILE
  const rows = await getEntriesPage({ offset: id * PER_FILE, limit: PER_FILE });
  return rows.map((r) => ({
    url: `https://example.com/${r.category}/${r.slug}`,
    lastModified: r.contentUpdatedAt,
  }));
}

The files are served at /places/sitemap/0.xml, /places/sitemap/1.xml and so on. Only include rows that render a 200. A sitemap full of 404s wastes the crawls you asked for.

The sitemap index

The Next.js docs don’t describe automatic sitemap index output. A commenter on GitHub issue #61969 found no way to generate one and hardcoded it instead. Treat “you build it yourself” as likely. A route handler that writes <sitemapindex> from the same count works.

Google’s large sitemaps docs set three rules for the index:

Reference it from robots.txt or submit it in Search Console. Google retired the ping endpoint in June 2023.

How should lastmod and revalidation be set so Google trusts the sitemap?

Google uses <lastmod> only if it is “consistently and verifiably accurate,” and it should reflect “the last significant update”: main content, structured data or links. The Next.js sitemap.ts example uses lastModified: new Date(), which means every URL claims a change on every request. Don’t copy it.

Store a content_updated_at column that your pipeline bumps only when fields that render on the page change. Then trigger revalidation from the same event with revalidateTag(\entry:${category}:${slug}`, ‘max’)`. The page and its lastmod then change together. That pairing is my recommendation; the docs don’t spell it out.

Also note that the sitemap.ts docs say the route is cached by default unless it uses a request-time API. Make sure it regenerates often enough to pick up new rows. The indexing hub covers how to read the results in Search Console.

How do canonicals and generateMetadata work for templated pages with filters?

Set metadataBase once in the root layout and build each canonical from params only. Per the generateMetadata docs:

For filters, Google’s faceted navigation doc is blunter than most SEO guides. It says canonical “may” reduce crawling over time but is “generally less effective” than robots.txt. Block filters you don’t want indexed in robots.txt. For filters you do index, use & separators, keep a consistent order, and return 404 for empty or nonsensical combinations.

When is static pre-rendering the wrong choice for a programmatic site?

On crawl budget: Google’s guide is written for sites with 1M+ pages changing weekly, or 10,000+ changing daily. At 50,000 mostly stable pages you may be below that line, but soft 404s and 5xx errors still cost you crawls.

What I’d check in review