Sourceframe

Workflow pattern / Extraction

Build a public real-estate listing intelligence workflow

Track publicly listed properties as structured records with explicit units, currencies, and listing history.

Advanced / Dataset / Daily

The problem

Listing pages vary by market and platform, use inconsistent units, and disappear without notice, making trend work unreliable.

Who it is for

Research teams working with publicly listed property information.

Teams: Research / Investment / Operations

Inputs

  • Permitted public listing sources
  • A property schema
  • Unit and currency rules
  • Retention and removal policy

Conceptual process

  1. 01Scope

    Confirm which sources permit automated access and record the policy per source.

  2. 02Enumerate

    Collect listing URLs within the defined geography.

  3. 03Extract

    Capture price, area with unit, type, and listing status.

  4. 04History

    Append changes rather than overwriting, so relistings are visible.

  5. 05Validate

    Range-check price per area and quarantine implausible records.

Flow diagram

  1. 01Sources
  2. 02Enumerate
  3. 03Extract
  4. 04Append history
  5. 05Validate
  6. 06Dataset

Example output

Illustrative output
{  "listing_id": "EX-77120",  "price": { "amount": 480000, "currency": "CAD" },  "area": { "value": 92, "unit": "sqm" },  "property_type": "apartment",  "status": "active",  "history": [{ "observed": "2026-08-02", "price": 495000 }],  "source_url": "https://example.com/listings/ex-77120"}

Data-quality considerations

  • Area units differ by market; never normalize silently.
  • A removed listing is not a sale — record status changes as observations.
  • Respect each source's stated access policy and rate limits.

Failure modes

  • Relisted properties counted as new inventory.
  • Price history lost by overwriting records.
  • Currency inferred from geography rather than from the page.

Suggested architecture

  • Source policy registry
  • Listing enumerator
  • Schema extractor
  • Append-only history store
  • Validation and quarantine

What to test first

  1. 01Track ten listings through one status change each.
  2. 02Confirm unit handling across two markets with different conventions.

Related reading