Tutorial · Research Workflows

How to Build a Company Research Assistant From a URL

A repeatable profile built from a company's own public pages.

Independent Content Strategist & Product-Marketing Writer · Published Aug 5, 2025

Primary audience: Research, intelligence, and editorial teams
Also useful for: AI application developers
Tone: Educational
Reading time: 2 min
Published: Aug 5, 2025

Direct answer

Build a company research assistant by crawling a shallow scope from the homepage, classifying pages by type, and running a type-specific extractor for each. The output is a profile with per-field provenance and an explicit not-found list. The discipline is refusing to infer: headcount, revenue, and funding are not derivable from a company's own marketing pages.

In active development — preview and full outline below

On this page

The classifier is the product
Definition: what a company's own pages can evidence
What the assistant should refuse to output

The classifier is the product

The valuable part of this workflow is not the extraction; it is the page-type classifier. Pricing pages, about pages, careers pages, and documentation each answer different questions. Treating them as one undifferentiated corpus produces a profile made of marketing adjectives — "leading", "trusted", "enterprise-grade" — which is exactly the output nobody can use.

Definition: what a company's own pages can evidence

A company site is a primary source for what the company claims and a weak source for what is true. "We support SOC 2" on a trust page evidences the claim and its date, not the certificate. Build the profile to record claims with their source, and let the reader decide what to verify.

Page type to extractor, and the limit of each

Page type	Fields worth extracting	What it cannot evidence
Pricing	Tier names, published prices, seat/usage units, contract terms	Actual paid price; enterprise deals are unlisted
Product / features	Named capabilities, integrations, supported platforms	Maturity or quality of any of them
Documentation	Limits, auth methods, API surface, changelog cadence	Roadmap or internal priorities
Careers	Open roles, team names, locations	Headcount — a proxy at best, and often stale
About / press	Stated positioning, named leadership, stated founding year	Funding, revenue, or customer counts
Customers / logos	Named references the company publishes	Whether those accounts are current

What the assistant should refuse to output

Headcount, revenue, funding, or valuation — not present on the site, and inferring them is fabrication.
Customers not named on a retrieved page, however plausible the logo grid looked.
A capability judgement ("strong at X") — record the claim and its excerpt instead.
Any field with no source_url. If provenance is missing, the field is missing.

Full outline

Sections planned for this guide

01Scoping the crawl
- — Depth and page caps
- — Subdomains and regional sites
- — What to exclude
02Page-type classification
- — The types worth distinguishing
- — Signals that identify each
- — Handling unclassifiable pages
03Per-type extraction
- — What each page type can evidence
- — Excerpt requirements
- — Marketing language versus capability
04Reconciliation
- — When pages disagree
- — Keeping both values with sources
- — Recency as a weak tiebreak
05What to refuse
- — Headcount, revenue, funding
- — Inferred customers
- — Anything not on a retrieved page

Practical takeaway

Classify page type before extracting; one extractor for all pages performs badly.
A company's pages are authoritative about claims, not outcomes.
An explicit not_found list is what makes the profile honest.
Re-run before use — profiles go stale quietly.