# Programmatic SEO with AI: using the Grok API as a pipeline worker

> How to use the Grok API for programmatic SEO: cleaning rows, schema-checked JSON fields, QA and dedup checks, and real costs for building 10,000 pages.

Maciej Wiatr, 2026-10-31. https://maciejwiatr.com/blog/programmatic-seo-with-ai-grok-api-pipeline

Programmatic SEO with AI usually means asking a model to write thousands of pages. That approach is what Google's spam updates have been removing. This post uses xAI's Grok API differently. Grok does one narrow job at each step of the pipeline: it cleans and enriches dataset rows, fills short page fields as schema-checked JSON, and runs a QA pass that blocks thin or near-duplicate pages before they reach the sitemap. I've written before about [Grok as a crawler hitting your pages](https://maciejwiatr.com/blog/programmatic-seo-ai-agent-grok-bot-traffic). This post is about Grok as the worker in the build. It covers costs for 10,000 pages, where a human has to stay involved, and when to skip the model entirely.

## Does programmatic SEO still work?

Yes, when each page carries data the next page doesn't. Google's [spam policies](https://developers.google.com/search/docs/essentials/spam-policies) target pages made "for the primary purpose of manipulating search rankings and not helping users… no matter how it's created." One of the listed examples is "Using generative AI tools… to generate many pages without adding value for users."

The 2026 enforcement was heavy. The [Search Status Dashboard](https://status.search.google.com/products/rGHU1u87FJnkP6W2GwMi/history) shows spam updates in March, June, August and September 2026, plus core updates in March and May. In [Glenn Gabe's August 2026 case studies](https://www.gsqi.com/marketing-blog/august-2026-google-spam-update-case-studies/), one site combined programmatic country pages with fully AI-generated YMYL content and lost rankings for 200K+ queries. A programmatic Amazon affiliate site lost 14K+. I went through the full timeline in [Is programmatic SEO dead?](https://maciejwiatr.com/blog/is-programmatic-seo-dead-google-updates).

![Google Search Status Dashboard table of 2026 ranking incidents, listing spam, core and Discover updates with dates and durations.](https://maciejwiatr.com/assets/blog/programmatic-seo-with-ai-grok-api-pipeline/shot-1.webp "The 2026 entries show spam updates in March, June, August and September, with core updates in March and May (status.search.google.com, Oct 2026)")

On the other side, a vendor case from [theStacc](https://thestacc.com/blog/programmatic-seo-case-study/) required at least 3 unique data points per page and reported 87% indexing within 90 days. It is unverified vendor data, but it lines up with Google's policy: the model can help as long as the data is doing the work.

## How to implement programmatic SEO with AI

The order is the same with or without a model. You need a dataset with real value in each row, a keyword pattern, one template, a pilot batch, and paced releases. Pacing affected indexing in theStacc's case: the normal rate of 25 to 30 pages a week indexed well, and a single 80-page week cut that batch's indexing to 41%.

The pipeline below shows where Grok sits and where code takes over:

```mermaid
flowchart LR
  A[Raw dataset rows] --> B[Grok cleans rows]
  B --> C[Grok grounded enrichment]
  C --> D[Code validates values]
  D --> E[Grok writes JSON fields]
  E --> F[Deterministic template render]
  F --> G[Dedup and QA]
  G --> H[Sitemap or hold]
```

## Which steps should Grok handle, and which stay deterministic?

My rule is that Grok handles judgement and code handles anything a reader could check against the source.

| Grok handles | Code handles |
|---|---|
| Normalising messy fields (categories, names, formats) | URLs and slugs |
| Grounded enrichment with citations | Numbers, prices, specs |
| Short per-page fields as constrained JSON | Titles built from data |
| Rubric-based QA scoring | Canonicals, internal links |
| | Sitemap inclusion, dedup math |

The reason comes from Google's own [generative AI guidance](https://developers.google.com/search/docs/fundamentals/using-gen-ai-content): models "don't retrieve facts, but predict a likely sequence of words." The same page says to check the output, including `<title>` and structured data. If a title can be assembled from the row, assemble it in code.

For cleaning, use function calling. xAI's [structured outputs docs](https://docs.x.ai/developers/model-capabilities/text/structured-outputs) say tool arguments "strictly conform" to the schema. Define a `normalize_row` tool whose arguments are the cleaned fields. For enrichment, the [web search tool](https://docs.x.ai/developers/tools/web-search) accepts `allowed_domains` with up to 5 domains. Point it at the official sources for your niche, then store `response.citations` next to every enriched value.

## How do structured outputs keep AI fields tied to the source data?

Use `response_format` with `type: "json_schema"`, and use the schema to limit what the model can say. Here is a sketch for a directory listing:

```json
{
  "type": "object",
  "properties": {
    "category": { "type": "string", "enum": ["dentist", "orthodontist", "oral_surgeon"] },
    "listing_id": { "type": "string", "pattern": "^[a-z]{2}-[0-9]{5}$" },
    "summary": { "type": "string", "maxLength": 400 },
    "facts": {
      "type": "array",
      "maxItems": 8,
      "items": {
        "type": "object",
        "properties": {
          "claim": { "type": "string", "maxLength": 200 },
          "source_field": { "type": "string", "enum": ["opening_hours", "services", "languages", "rating"] }
        },
        "required": ["claim", "source_field"]
      }
    }
  },
  "required": ["category", "listing_id", "summary", "facts"]
}
```

An `enum` means the category can only come from your dataset's own list. `pattern` locks the ID format. Every claim has to name the row field it came from. `additionalProperties` defaults to false, so the model can't add fields of its own.

The schema has limits. xAI enforces `maxLength` only up to 2,048 and `maxItems` up to 256. `pattern` supports a subset of ECMA-262 with no lookaround or backreferences, and conditionals like `if`/`then`/`else` are best-effort. The docs say plainly: "We recommend validating if strict conformance is required."

A schema guarantees the shape of the output. It doesn't guarantee the content is true. After every call, I would validate in code with Zod or Pydantic and then diff the output against the row. Each `source_field` must be non-empty in the row, and any number in a `claim` must appear in that field's value. Pages that fail don't render.

## What does it cost to generate 10,000 pages with the Grok API?

[xAI's model page](https://docs.x.ai/docs/models) lists grok-4.6 at $2 per 1M input tokens, $0.50 per 1M cached input and $6 per 1M output, with a 500k context. Those rates apply to prompts under 200k tokens and double above that. [xAI's pricing page](https://docs.x.ai/developers/pricing) gives a 20% Batch API discount on "input, output, cached, and reasoning tokens," which works out to $1.60, $0.40 and $4.80. `web_search` costs $5 per 1,000 calls. Some third-party guides claim 4.6 gets no batch discount, but the official page says it does. Check the page before you budget.

![xAI API pricing page showing a Text API table of Grok model prices per million tokens for input, cached input and output.](https://maciejwiatr.com/assets/blog/programmatic-seo-with-ai-grok-api-pipeline/shot-2.webp "Short-context rates per million tokens, for example grok-4.7 at $2.00 input, $0.50 cached and $6.00 output; check the live page for Batch API discounts (docs.x.ai, Oct 2026)")

The token counts below are my assumptions, not measurements:

| Step | Tokens per page (assumed) | Real-time per 1,000 | Batch per 1,000 |
|---|---|---|---|
| Enrichment, 1 web search | 6,000 in, 800 out | $21.80 | $18.44 |
| Field generation | 2,000 cached + 1,000 fresh in, 1,500 out | $12.00 | $9.60 |
| QA pass | 1,500 cached + 1,000 fresh in, 500 out | $5.75 | $4.60 |
| **Total** | | **$39.55** | **$32.64** |

That comes to roughly $396 real-time or $326 in batch for 10,000 pages. I left the search fee undiscounted in the batch column because xAI's discount wording only covers tokens. The enrichment input figure is the shakiest number here, because I found no data on how many tokens search results add.

The numbers move in three ways:

- **Caching is best case.** xAI's [caching guide](https://docs.x.ai/developers/advanced-api-usage/prompt-caching/best-practices) says hits aren't guaranteed. Put static content first, send `x-grok-conv-id` or `prompt_cache_key`, and check `cached_tokens` in the usage data. The [cookbook](https://docs.x.ai/cookbook/examples/prompt-caching.md) warns that a timestamp at the top of the system prompt breaks caching. None of the docs I found say whether caching works inside batches. Without it, batch generation costs $12 per 1,000, the same as cached real-time.
- **Reasoning can't be turned off on grok-4.6.** The [reasoning docs](https://docs.x.ai/developers/model-capabilities/text/reasoning) list a default effort of `high`, and reasoning tokens are billed. Set `reasoning_effort: low` for cleaning and QA calls.
- **The model choice will date.** grok-4.7 has the same prices and context as 4.6. grok-4.3 costs $1.25, $0.20 and $2.50 with a 1M context. I would test 4.3 on the QA pass first.

Batch fits this job well. One file holds up to 50,000 requests, each with a unique `custom_id`, according to the [Batch API guide](https://docs.x.ai/docs/guides/batch-api). Most batches finish within 24 hours, but that is "best effort and not guaranteed." Batch requests also skip the real-time limits of 150 requests per second and 50M tokens per minute. One catch for enrichment: server-side web search runs inside a batch, but your own function tools return `tool_calls`, and each extra round needs a new batch.

For scale, theStacc's vendor case reports $80 per page with human editing. Tokens cost about 4 cents a page, so review is where the money goes.

## How do you flag thin or near-duplicate pages before publishing?

Run the checks on rendered HTML text, after the template has filled in, and run them across all pages:

1. **Data floor.** Count the unique, non-empty data fields per page. theStacc's minimum was 3.
2. **SimHash.** [Manku et al.](https://www.www2007.cpsc.ucalgary.ca/papers/paper215.pdf) tested on 8 billion pages and found "64-bit simhash fingerprints and k = 3 are reasonable." Pages within 3 bits of each other count as near-duplicates.
3. **Shingles or MinHash.** The [Stanford IR book](https://nlp.stanford.edu/IR-book/html/htmledition/near-duplicates-and-shingling-1.html) uses Jaccard similarity over 4-word shingles. [Lee et al.](https://arxiv.org/pdf/2107.06499) confirmed MinHash LSH candidates at Jaccard above 0.8.
4. **Paraphrase check.** MinHash only measures word overlap, so it [misses paraphrases](https://metricgate.com/docs/minhash-near-duplicate-detection/), which is exactly what AI copy tends to produce. My suggestion is to compare the data fields behind two pages, not just their text. If the fields match, the pages are duplicates however different the wording looks.
5. **Grok QA rubric.** Score each page against a fixed rubric at low reasoning effort.

Failed pages stay out of the sitemap or get `noindex`. Google's [crawl budget guide](https://developers.google.com/search/docs/crawling-indexing/large-site-managing-crawl-budget) says to "eliminate duplicate content" and warns that soft 404s keep being crawled and "waste your budget." For more on what makes a page unique, see [shared blocks vs unique page data](https://maciejwiatr.com/blog/programmatic-seo-template-unique-content).

## Where a human stays in the loop

- **Rubric calibration.** Hand-check a sample before trusting the QA scores.
- **YMYL topics.** Health and finance pages need expert review. Gabe's 200K-query case was AI content on an ultra-YMYL site.
- **Metadata and labels.** Google suggests disclosing how automated content was made, and Merchant Center requires AI-generated product titles and descriptions to be labelled.
- **Pruning.** In one vendor case from [Vizup](https://www.tryvizup.com/blog/programmatic-seo-trends-2026), 73% of a 12,000-page build had zero impressions after 6 months. Someone has to decide whether to delete, merge or enrich those pages.

## When is AI generation the wrong choice for programmatic pages?

- **The row has too little data.** Generated filler only rewords it, which matches Google's "synonymizing" and "stitching" examples.
- **The values are numbers.** Prices, specs and rates should render straight from the data. A model can only introduce errors there.
- **YMYL pages without expert review.**
- **The pages target slight query variations.** Google classes that as doorway abuse, and better copy doesn't change the classification.

My recommendation: start with the deterministic render. Add Grok only for fields your data can't supply on its own, and let the QA pass block pages from publishing. A model that only rewrites rows adds risk without adding value.