Technical guide · Structured Data

How to Extract Product Listings Into Structured JSON

Pagination, variants, currencies, and the fields that quietly go wrong.

Independent Content Strategist & Product-Marketing Writer · Published Oct 8, 2025

Primary audience: AI engineers and technical founders
Also useful for: Data and operations teams using public-web information
Tone: Educational
Reading time: 2 min
Published: Oct 8, 2025

Direct answer

Extract product listings by enumerating detail-page URLs from listing pagination, then extracting a schema per detail page with currency and unit captured as fields. The recurring errors are variant prices captured as base prices, promotional prices overwriting list prices, and out-of-stock items dropped instead of recorded. Validate ranges, quarantine failures, and track extraction rate per template.

In active development — preview and full outline below

Why catalog data is wrong in ways that look right

Catalog extraction looks like the easiest structured-data task and produces the most subtly wrong datasets. Almost every failure still yields a valid-looking record: a plausible number in the price field that happens to be the largest variant, or a promotional price captured on a day a promotion was running. Nothing throws. The dataset passes review because every row has a number in it.

Two decisions prevent most of this, and both are made before any extraction runs: what one row represents, and which fields are allowed to be null.

Definition: what is one row?

A product row can mean a product, a variant, or an offer. A notebook sold in two rulings and three sizes is one product, six variants, and — across two regional storefronts — twelve offers. Pick one and state it in the schema. Most price-comparison errors are two teams comparing rows that mean different things.

Field-level failure modes and the assertion that catches each

Field	What goes wrong	Assertion
price	Captures the selected variant, not the row's variant	Variant string is non-null whenever the page exposes a selector
list_price	Promotional price overwrites it	price ≤ list_price; flag when they are equal on a page showing a discount badge
currency	Inferred from the symbol; $ is ambiguous	Currency must come from a code on the page or from the storefront config, never from a glyph
availability	Out-of-stock rows dropped	Row count per category is stable run to run; absence is `out_of_stock`, not a missing row
sku	Falls back to the URL slug	SKU matches the site's own format pattern, else null
source_url	Points at the listing page	URL resolves to a detail page and matches the canonical link

Illustrative record shapejson · 9 lines

{  "sku": "EX-1042",  "price": { "amount": 18.5, "currency": "USD" },  "list_price": { "amount": 22.0, "currency": "USD" },  "variant": "ruled / A5",  "availability": "in_stock",  "observed_at": "2026-09-14T10:02:00Z",  "source_url": "https://example.com/products/ex-1042"}

Full outline

Sections planned for this guide

01Enumerating the catalog
- — Pagination patterns and where they end early
- — Category overlap and deduplication
- — Detecting the last page reliably
02Variants
- — Which variant a record represents
- — Variant-specific prices and availability
- — When variants deserve their own records
03Prices
- — List versus promotional price
- — Currency as a field
- — Tax-inclusive and regional differences
04Availability
- — Recording absence rather than dropping
- — Status vocabularies across sites
- — Observation time versus stock time
05Validation
- — Range checks and outlier quarantine
- — Per-template extraction rate
- — Sampling for review

Practical takeaway

Enumerate first, extract second — mixing the two hides pagination bugs.
Variant selection changes the price; decide which variant the record represents.
An out-of-stock product is a record, not a gap.
Track extraction rate per template so a redesign is visible within a day.