Workflow pattern / Extraction
Create a local-business research dataset
Build a bounded dataset of public business information for a defined geography and category.
Introductory / Dataset / Weekly
The problem
Local information is scattered across directories and individual sites, with inconsistent formats and heavy duplication.
Who it is for
Teams needing a bounded local dataset for research or operations planning.
Teams: Research / Operations / Growth
Inputs
- Geography and category definition
- Directory seeds
- A business schema
- Deduplication keys
Conceptual process
01Seed
Collect candidates from public directories and association listings.
02Verify
Visit each business's own site to confirm details where one exists.
03Extract
Capture name, category, address, hours, and contact, with sources.
04Deduplicate
Match on normalized name plus address; keep both records when unsure.
05Bound
State clearly which sources were used and what the dataset excludes.
Flow diagram
- 01Directories
- 02Candidates
- 03Verify on site
- 04Extract
- 05Dedupe
- 06Dataset
Example output
{ "name": "Example Bindery", "category": "Print and bindery", "address": "12 Example Street", "hours_source": "https://example.com/visit", "confidence": "verified on own site", "duplicates_merged": 2, "retrieved_at": "2026-09-14"}Data-quality considerations
- Directory data is frequently stale; a business's own site wins when both exist.
- Avoid collecting personal data that is not clearly published for business contact.
- State coverage bounds — a local dataset is never complete.
Failure modes
- Closed businesses retained because their directory entry persists.
- Chains merged into a single record by name matching.
- Category assignment inherited from a directory's own inconsistent taxonomy.
Suggested architecture
- Directory seeds
- Site verifier
- Schema extractor
- Dedupe on name+address
- Bounded dataset export
What to test first
- 01Verify twenty records by hand and count the stale ones.
- 02Check chain handling explicitly.
Related reading