Everything, in one index.

Technical guide · Intermediate

A practical guide to discovery, retrieval, clean context, structured extraction, and source traceability.

AI application developers

Foundational guide · Introductory

Why clean, structured text often matters more than raw HTML in AI workflows.

AI application developers

Decision guide · Introductory

Choose the right web-data approach for research, RAG, monitoring, and agent workflows.

AI engineers and technical founders

Tutorial · Advanced

A source-first workflow for turning open-web information into accountable AI answers.

Research, intelligence, and editorial teams

Tutorial · Intermediate

Turn product documentation into a useful, maintainable knowledge source.

AI application developers

Workflow playbook · Intermediate

Separate signal from noise when product pages, policies, prices, and documentation update.

Research, intelligence, and editorial teams

Technical guide · Advanced

Use schemas to transform inconsistent pages into dependable records.

AI engineers and technical founders

Tutorial · Intermediate

Track positioning, pricing, launches, messaging shifts, and source-backed market signals.

Research, intelligence, and editorial teams

Foundational guide · Intermediate

Why usable source material shapes AI quality more than most product teams expect.

AI application developers

Strategic guide · Intermediate

A practical, skeptical framework for tracking whether your product appears in AI-generated answers.

Growth and content leaders at developer-tool companies

Foundational guide · Intermediate

The layer between a user question and a model call — source selection, retrieval, normalization, structured context, and provenance.

AI application developers

Tutorial · Advanced

One ingest path, many sources: standardising how external content enters internal systems.

AI application developers

Technical guide · Intermediate

Pagination, variants, currencies, and the fields that quietly go wrong.

AI engineers and technical founders

Tutorial · Advanced

Evidence-based comparison where 'not established' is a valid answer.

Research, intelligence, and editorial teams

Strategic guide · Introductory

From a URL list to a queryable table of pages, topics, and structure.

Growth and content leaders at developer-tool companies

Tutorial · Intermediate

A repeatable profile built from a company's own public pages.

Research, intelligence, and editorial teams

Tutorial · Introductory

A short, defensible daily read built from a defined source set.

Research, intelligence, and editorial teams

Tutorial · Intermediate

A category map where every placement has a source and a reason.

Research, intelligence, and editorial teams

Technical guide · Intermediate

What to carry alongside content, and what it lets your product say.

AI engineers and technical founders

Foundational guide · Introductory

What these systems actually do well, and where the claims outrun the engineering.

AI application developers

Decision guide · Introductory

A cost-and-fragility view of the four ways to get web data.

AI engineers and technical founders

Field note · Introductory

“Give the agent web access” is a capability statement pretending to be a requirement.

Product managers building AI features

Field note · Introductory

A ranked list answers “where might this be?” Products usually need “what does it say, and as of when?”

AI application developers

Field note · Introductory

A feed of page changes is raw material. Intelligence is the part where a named person can act.

Research, intelligence, and editorial teams

Field note · Introductory

Guides, docs, and templates are where evaluation and implementation actually happen.

Growth and content leaders at developer-tool companies

Field note · Introductory

Retrieval is a search problem. Usability is a structure problem. They fail in different places.

AI application developers

Field note · Introductory

Citations are an interface decision long before they are a trust feature.

Product managers building AI features

Documentation · Intermediate

A reference for schema design, validation, retries, review thresholds, and error handling in page-to-record extraction.

AI engineers and technical founders

Documentation · Intermediate

What to record about a retrieved page so a claim can be checked months later — and what not to store.

AI application developers

Documentation · Introductory

The editorial standard used for guides, documentation, and templates on this site — written to be copied and adapted.

Developer advocates and technical writers

Glossary explainer · Introductory

The record connecting an output back to the page, moment, and sentence it came from.

AI application developers

Glossary explainer · Introductory

Deciding whether a page changed in a way that matters, rather than whether its bytes changed.

Research, intelligence, and editorial teams

Glossary explainer · Introductory

Turning a page into typed records against a schema you defined first.

AI application developers

Research note · Intermediate

Most AI products that reason about the outside world fail for an unglamorous reason: the material they reason over is incomplete, stale, or structurally unusable. This note argues

Product managers building AI features

Research note · Intermediate

A source-linked answer is not an answer with links appended. This note breaks a well-formed answer into its parts — claim, support, provenance, confidence, and refusal — and argues

Product managers building AI features

Research note · Intermediate

Raw HTML is a rendering instruction set that happens to contain text. This note examines what is lost and gained when a page is normalized to structured Markdown, and identifies th

Product managers building AI features

Research note · Intermediate

The first version of a web-data pipeline is usually a weekend. The cost arrives afterwards, in silent breakage, template drift, and the operational question of who notices when a s

Product managers building AI features

Research note · Intermediate

Agents interact with a web that was designed for human readers and search crawlers. This note sets out what an agent actually requires from a page, where current conventions fall s

Product managers building AI features

Research note · Intermediate

Debugging a web-data pipeline is easier when failures have names. This note proposes a working taxonomy across five layers — access, retrieval, normalization, extraction, and inter

Product managers building AI features

Research note · Intermediate

Search solves location. Most AI products fail at the step after location: turning a set of plausible pages into material a system can act on. This note separates the two problems a

Product managers building AI features

Research note · Intermediate

Reliability in a web-enabled AI product is not an average of component accuracies; it is determined by how the system behaves when a component fails. This note traces the path from

Product managers building AI features

Research note · Intermediate

Automated research is good at breadth and weak at judgement. This note sets out the specific judgements that resist automation — source credibility, conflicting evidence, and the d

Product managers building AI features

Research note · Intermediate

Evaluating a web-enabled workflow with a single accuracy number hides where it is actually weak. This note proposes a five-dimension evaluation frame — coverage, freshness, fidelit

Product managers building AI features

Workflow playbook · Advanced

Answer open questions from current public sources, with every claim linked to the passage that supports it.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Crawl a documentation set, clean it, chunk it on structure, and keep it current without rebuilding from scratch.

AI application developers

Workflow playbook · Intermediate

Watch a defined set of public pages and alert only when a change could plausibly change a decision.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Start from a single homepage and assemble a structured profile from the company's own public pages.

Research, intelligence, and editorial teams

Workflow playbook · Advanced

Turn listing and detail pages into validated records with units, currencies, and source links intact.

AI engineers and technical founders

Workflow playbook · Intermediate

Assemble a short, source-linked daily brief from a defined set of public sources on a defined topic.

Research, intelligence, and editorial teams

Workflow playbook · Advanced

Compare vendors on declared criteria using each vendor's own public pages, with disagreement preserved.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Assemble a map of who exists in a category and how they describe themselves, using only their own public pages.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Track a narrow set of high-consequence pages where a single sentence can change a decision.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Understand how a topic is being covered publicly, where the gaps are, and which formats recur.

Research, intelligence, and editorial teams

Workflow playbook · Intermediate

Convert public job postings into a structured dataset about declared skills, roles, and locations.

AI engineers and technical founders

Workflow playbook · Introductory

Build a bounded dataset of public business information for a defined geography and category.

AI engineers and technical founders

Workflow playbook · Advanced

Track publicly listed properties as structured records with explicit units, currencies, and listing history.

AI engineers and technical founders

Workflow playbook · Advanced

Support a purchase decision with documented, source-linked evidence against written requirements.

Research, intelligence, and editorial teams

Workflow playbook · Advanced

A general ingest pipeline that turns an approved source list into a maintained, queryable knowledge base.

AI application developers

Checklist · Intermediate

Run this before an automated research output is shown to anyone who will act on it. It is a review aid, not a certification.

Research, intelligence, and editorial teams

Checklist · Intermediate

A pre-ingest review of whether a content set is worth indexing yet. Directional, not a formal audit.

Research, intelligence, and editorial teams

Checklist · Advanced

Validation rules to apply before extracted records reach a database or a downstream model.

Research, intelligence, and editorial teams

Template · Introductory

Fill this in per monitored source before adding it to a monitoring run. Copy the structure into your own tracker.

Research, intelligence, and editorial teams

Template · Introductory

A short review record to attach to any AI output that will be published or acted on.

Research, intelligence, and editorial teams

Template · Introductory

Fill this in before anyone writes retrieval code. Most disagreements about model quality turn out to be unanswered questions on this page. It is a planning aid, not a specification standard.

Research, intelligence, and editorial teams

Template · Introductory

A structured way to find out whether a web-enabled AI workflow is worth building, before estimating it. Work through it with the person who does the task manually today.

Research, intelligence, and editorial teams