Programmatic SEO with AI: using the Grok API as a pipeline worker

Programmatic SEO with AI usually means asking a model to write thousands of pages. That approach is what Google’s spam updates have been removing. This post uses xAI’s Grok API differently. Grok does one narrow job at each step of the pipeline: it cleans and enriches dataset rows, fills short page fields as schema-checked JSON, and runs a QA pass that blocks thin or near-duplicate pages before they reach the sitemap. I’ve written before about Grok as a crawler hitting your pages. This post is about Grok as the worker in the build. It covers costs for 10,000 pages, where a human has to stay involved, and when to skip the model entirely.

Does programmatic SEO still work?

Yes, when each page carries data the next page doesn’t. Google’s spam policies target pages made “for the primary purpose of manipulating search rankings and not helping users… no matter how it’s created.” One of the listed examples is “Using generative AI tools… to generate many pages without adding value for users.”

The 2026 enforcement was heavy. The Search Status Dashboard shows spam updates in March, June, August and September 2026, plus core updates in March and May. In Glenn Gabe’s August 2026 case studies, one site combined programmatic country pages with fully AI-generated YMYL content and lost rankings for 200K+ queries. A programmatic Amazon affiliate site lost 14K+. I went through the full timeline in Is programmatic SEO dead?.

On the other side, a vendor case from theStacc required at least 3 unique data points per page and reported 87% indexing within 90 days. It is unverified vendor data, but it lines up with Google’s policy: the model can help as long as the data is doing the work.

How to implement programmatic SEO with AI

The order is the same with or without a model. You need a dataset with real value in each row, a keyword pattern, one template, a pilot batch, and paced releases. Pacing affected indexing in theStacc’s case: the normal rate of 25 to 30 pages a week indexed well, and a single 80-page week cut that batch’s indexing to 41%.

The pipeline below shows where Grok sits and where code takes over:

flowchart LR
  A[Raw dataset rows] --> B[Grok cleans rows]
  B --> C[Grok grounded enrichment]
  C --> D[Code validates values]
  D --> E[Grok writes JSON fields]
  E --> F[Deterministic template render]
  F --> G[Dedup and QA]
  G --> H[Sitemap or hold]

Which steps should Grok handle, and which stay deterministic?

My rule is that Grok handles judgement and code handles anything a reader could check against the source.

The reason comes from Google’s own generative AI guidance: models “don’t retrieve facts, but predict a likely sequence of words.” The same page says to check the output, including <title> and structured data. If a title can be assembled from the row, assemble it in code.

For cleaning, use function calling. xAI’s structured outputs docs say tool arguments “strictly conform” to the schema. Define a normalize_row tool whose arguments are the cleaned fields. For enrichment, the web search tool accepts allowed_domains with up to 5 domains. Point it at the official sources for your niche, then store response.citations next to every enriched value.

How do structured outputs keep AI fields tied to the source data?

Use response_format with type: "json_schema", and use the schema to limit what the model can say. Here is a sketch for a directory listing:

{
  "type": "object",
  "properties": {
    "category": { "type": "string", "enum": ["dentist", "orthodontist", "oral_surgeon"] },
    "listing_id": { "type": "string", "pattern": "^[a-z]{2}-[0-9]{5}$" },
    "summary": { "type": "string", "maxLength": 400 },
    "facts": {
      "type": "array",
      "maxItems": 8,
      "items": {
        "type": "object",
        "properties": {
          "claim": { "type": "string", "maxLength": 200 },
          "source_field": { "type": "string", "enum": ["opening_hours", "services", "languages", "rating"] }
        },
        "required": ["claim", "source_field"]
      }
    }
  },
  "required": ["category", "listing_id", "summary", "facts"]
}

An enum means the category can only come from your dataset’s own list. pattern locks the ID format. Every claim has to name the row field it came from. additionalProperties defaults to false, so the model can’t add fields of its own.

The schema has limits. xAI enforces maxLength only up to 2,048 and maxItems up to 256. pattern supports a subset of ECMA-262 with no lookaround or backreferences, and conditionals like if/then/else are best-effort. The docs say plainly: “We recommend validating if strict conformance is required.”

A schema guarantees the shape of the output. It doesn’t guarantee the content is true. After every call, I would validate in code with Zod or Pydantic and then diff the output against the row. Each source_field must be non-empty in the row, and any number in a claim must appear in that field’s value. Pages that fail don’t render.

What does it cost to generate 10,000 pages with the Grok API?

xAI’s model page lists grok-4.6 at $2 per 1M input tokens, $0.50 per 1M cached input and $6 per 1M output, with a 500k context. Those rates apply to prompts under 200k tokens and double above that. xAI’s pricing page gives a 20% Batch API discount on “input, output, cached, and reasoning tokens,” which works out to $1.60, $0.40 and $4.80. web_search costs $5 per 1,000 calls. Some third-party guides claim 4.6 gets no batch discount, but the official page says it does. Check the page before you budget.

The token counts below are my assumptions, not measurements:

That comes to roughly $396 real-time or $326 in batch for 10,000 pages. I left the search fee undiscounted in the batch column because xAI’s discount wording only covers tokens. The enrichment input figure is the shakiest number here, because I found no data on how many tokens search results add.

The numbers move in three ways:

• Caching is best case. xAI’s caching guide says hits aren’t guaranteed. Put static content first, send x-grok-conv-id or prompt_cache_key, and check cached_tokens in the usage data. The cookbook warns that a timestamp at the top of the system prompt breaks caching. None of the docs I found say whether caching works inside batches. Without it, batch generation costs $12 per 1,000, the same as cached real-time.
• Reasoning can’t be turned off on grok-4.6. The reasoning docs list a default effort of high, and reasoning tokens are billed. Set reasoning_effort: low for cleaning and QA calls.
• The model choice will date. grok-4.7 has the same prices and context as 4.6. grok-4.3 costs $1.25, $0.20 and $2.50 with a 1M context. I would test 4.3 on the QA pass first.

Batch fits this job well. One file holds up to 50,000 requests, each with a unique custom_id, according to the Batch API guide. Most batches finish within 24 hours, but that is “best effort and not guaranteed.” Batch requests also skip the real-time limits of 150 requests per second and 50M tokens per minute. One catch for enrichment: server-side web search runs inside a batch, but your own function tools return tool_calls, and each extra round needs a new batch.

For scale, theStacc’s vendor case reports $80 per page with human editing. Tokens cost about 4 cents a page, so review is where the money goes.

How do you flag thin or near-duplicate pages before publishing?

Run the checks on rendered HTML text, after the template has filled in, and run them across all pages:

1. Data floor. Count the unique, non-empty data fields per page. theStacc’s minimum was 3.
2. SimHash. Manku et al. tested on 8 billion pages and found “64-bit simhash fingerprints and k = 3 are reasonable.” Pages within 3 bits of each other count as near-duplicates.
3. Shingles or MinHash. The Stanford IR book uses Jaccard similarity over 4-word shingles. Lee et al. confirmed MinHash LSH candidates at Jaccard above 0.8.
4. Paraphrase check. MinHash only measures word overlap, so it misses paraphrases, which is exactly what AI copy tends to produce. My suggestion is to compare the data fields behind two pages, not just their text. If the fields match, the pages are duplicates however different the wording looks.
5. Grok QA rubric. Score each page against a fixed rubric at low reasoning effort.

Failed pages stay out of the sitemap or get noindex. Google’s crawl budget guide says to “eliminate duplicate content” and warns that soft 404s keep being crawled and “waste your budget.” For more on what makes a page unique, see shared blocks vs unique page data.

Where a human stays in the loop

• Rubric calibration. Hand-check a sample before trusting the QA scores.
• YMYL topics. Health and finance pages need expert review. Gabe’s 200K-query case was AI content on an ultra-YMYL site.
• Metadata and labels. Google suggests disclosing how automated content was made, and Merchant Center requires AI-generated product titles and descriptions to be labelled.
• Pruning. In one vendor case from Vizup, 73% of a 12,000-page build had zero impressions after 6 months. Someone has to decide whether to delete, merge or enrich those pages.

When is AI generation the wrong choice for programmatic pages?

• The row has too little data. Generated filler only rewords it, which matches Google’s “synonymizing” and “stitching” examples.
• The values are numbers. Prices, specs and rates should render straight from the data. A model can only introduce errors there.
• YMYL pages without expert review.
• The pages target slight query variations. Google classes that as doorway abuse, and better copy doesn’t change the classification.

My recommendation: start with the deterministic render. Add Grok only for fields your data can’t supply on its own, and let the QA pass block pages from publishing. A model that only rewrites rows adds risk without adding value.