Programmatic SEO deindexing recovery: prune, merge or rebuild

When thousands of templated pages slide into “Crawled, currently not indexed”, the usual reaction is to resubmit the sitemap and click Request Indexing. That rarely helps. Programmatic SEO deindexing recovery is mostly quality work: Google has decided that a template, or a pattern inside it, isn’t worth a place in the index. This post covers five things: how to confirm that, how to split the dropped URLs by template, what to prune, merge or improve, how to measure the recovery, and when to delete a template outright.

Why does Google crawl my programmatic pages but not index them?

Crawling and indexing are separate decisions. Google’s crawl budget guide says “not every page that is crawled will necessarily be indexed.” Google judges each page again after it fetches it.

Quality drives most of that judgement. Gary Illyes, quoted by Search Engine Journal, said index selection “largely depends on the page’s quality and the previously collected signals.” In a Search Off the Record episode reported in July 2026, John Mueller was asked whether non-indexing is a quality signal. He answered: “Sometimes. So it’s definitely the case if our systems are seriously worried about the quality of a website, they will reduce the number of pages that they index.”

He was more direct about programmatic sites. In September 2026 posts on Bluesky, he wrote that “Programmatic SEO like this often leads to a site that’s either spam, borderline spam, or low quality,” and that Google’s systems “have possibly lost faith in your site providing good value to users based on the old pages.” These quotes come from trade press coverage, not the original posts.

“Discovered, currently not indexed” is a different status. It means Google knows the URL but hasn’t crawled it yet. The Page indexing report docs say Google usually postpones the crawl because it “was expected to overload the site.” That status can point to a capacity problem. “Crawled” means Google fetched the page and chose not to keep it.

Does resubmitting a sitemap or Request Indexing fix it at scale?

No. Google’s own definition of the status ends with “no need to resubmit this URL for crawling.” The same page adds that “Indexing is never instant, even when you submit a crawl request directly.”

The tools also can’t handle the volume:

• Sitemaps are hints. Google’s sitemap docs say it may not download them at all. It ignores <priority> and <changefreq>, and it uses <lastmod> only when that value is “consistently and verifiably” accurate.
• Request Indexing has no published quota. Third-party reports range from about 5 to about 11 requests a day per property, and the figures conflict.
• The URL Inspection API reports status only. It allows 2,000 queries a day per property and can’t submit anything.

Most “resubmit and wait” advice, including the older 2022-era guides that still rank, was written for technical deindexing: a server outage, an accidental noindex or a hack. It still works for those cases. It does nothing about a quality decision.

Step 1: rule out technical causes first

Check redirects, noindex tags, robots rules, canonicals and rendering before you rewrite anything. In an October 2026 case, a 180,000-page product aggregator had only 4 URLs indexed even though Googlebot was crawling heavily. Mueller found that the URLs “all redirect to the homepage.” My recommendation is to run URL Inspection on a sample from every template and read the HTML Google rendered.

Step 2: tell never-indexed pages from dropped ones

Search Console doesn’t separate the two. Ahrefs notes that a page can move into “Crawled, currently not indexed” after it has been indexed. Indexing Insight argues that the status lumps both groups together. The fix differs: a template that was never indexed failed on day one, while a template that was dropped lost Google’s confidence over time.

Start with one sitemap per template. The Page indexing report can be filtered by a specific sitemap, and Google says submitting several sitemaps lets you track each one. Each file can hold up to 50,000 URLs or 50 MB uncompressed, so large templates need several files:

<?xml version="1.0" encoding="UTF-8"?>
<!-- /sitemaps/city-category/index.xml: submit this one file per template -->
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap><loc>https://example.com/sitemaps/city-category/1.xml</loc></sitemap>
  <sitemap><loc>https://example.com/sitemaps/city-category/2.xml</loc></sitemap>
</sitemapindex>

This setup also works around a reporting limit. The examples table is “limited to 1,000 rows,” so filtered counts per template are the only reliable totals on a large site.

Within each template, three signals help you split the two groups:

• Performance report history. If a URL had impressions at some point, it was indexed then. This is my inference; none of the sources states it outright.
• The URL Inspection API. Pull the coverage state and last crawl time for a sample of URLs.
• Last crawl date. Indexing Insight reports that pages not crawled for roughly 130 to 190 days drift out of the index. That comes from one vendor, and its cutoffs are inconsistent, so treat it as a hint.

Step 3: find the templates and data patterns that lost coverage

Compare the indexed ratio across template sitemaps. Then, inside each weak template, split the URLs by data pattern. Look for rows with only one or two fields filled, rows that duplicate a sibling page, and pages with empty listings.

A 512-page case on a vendor blog found that template pages with only one or two unique data points were the ones at risk. Pages with unique data, dense internal links and use-case sections held up. I haven’t seen it verified elsewhere, but it matches what Google says about quality.

Step 4: should you noindex, delete or merge thin pages?

Google’s crawl budget guide says to “return a 404 or 410 status code for permanently removed pages” and “Don’t use noindex” to manage crawl budget. Noindex removes pages from the index, but Google still requests them. On 404 versus 410, Mueller said in 2024 that the difference is minimal, so pick one and move on.

Does pruning help the remaining pages get indexed?

Possibly. Google’s 2022 helpful content post said removing unhelpful content could help other pages rank, and its later wording softened that to “might.” The core updates documentation also says “Deleting content is a last resort.” It adds that removing whole sections may be “a sign those sections were created for search engines first.”

The case studies point in both directions:

• A marketplace that noindexed about 600,000 zero-click pages. SEOCopilot reports clicks and impressions up about 30%. This is a single vendor case with no baseline.
• An ecommerce site that pruned and rewrote at the same time. Inflow reports sessions up 31.03%. The two changes can’t be separated, and the data is from around 2015 to 2016.
• A case where cutting about 80% of content backfired. In Orbit Media’s roundup, total keywords and traffic went down.

Mueller has also said that low traffic alone doesn’t mean a page needs removing. My recommendation is to prune by data pattern rather than by click threshold: remove the pages that have nothing of their own.

Step 5: add unique data to the pages you keep

According to TechWyse’s coverage, Mueller said that adding more generated filler to each page won’t fix the problem. In the July episode he said “we almost have to take into account the full experience on a page.” What counts is data the next URL doesn’t have, such as real counts, prices, dates and comparisons. The data hub covers where that data comes from.

How long does recovery take, and how do I measure it per template?

Google’s core updates doc says changes “could take several months,” that recovery may mean “waiting until the next core update,” and that “there’s no guarantee.” Mueller said it “tends to take time & significant effort.” Glenn Gabe’s August 2026 spam update cases cite a five-month recovery.

To measure each template:

• Snapshot indexed and not-indexed counts per template sitemap every week.
• Sample URLs with the Inspection API to see how coverage state and last crawl dates change.
• Track clicks and impressions per template with URL path filters in the Performance report.
• Keep one template you haven’t touched as a control.

Be careful with Validate Fix. It “typically takes up to about two weeks,” and it ends as soon as Google finds one remaining instance. When you filter by sitemap, validation covers only the URLs in that sitemap at the moment you request it. Put the fixed URLs in their own sitemap before you start validation. The measurement hub goes into the reporting setup.

When is a programmatic template not worth saving?

Google publishes no rule for this. These are practitioner signals:

• The pages swap only one or two fields.
• The pages show no demand across 90 days to 12 months.
• The template exists only to match keyword combinations, like the domain, technology and attribute pages Mueller criticised.
• A useful version would mean rebuilding the product. Gabe doubted that a programmatic Amazon affiliate site could recover without changing how the site is built and run.

Tailride is the extreme case. About 50,000 AI-generated pages were deindexed with no manual action. The founder told Search Engine Journal it “was thin content and probably duplicated.” The recovery involved a new domain and “Less pages + more quality.”

If a template meets two or more of these signals, I’d delete it with a 404 and spend the effort on the templates that have real data. For more on which page types still hold up, see what Google’s 2026 updates removed.