Research note · Research note 01 — current edition

The Web Context Problem

Independent Content Strategist & Product-Marketing Writer · Published Nov 4, 2025

Primary audience: Product managers building AI features
Also useful for: AI engineers and technical founders
Tone: Authoritative
Reading time: 2 min
Published: Nov 4, 2025

Abstract

Most AI products that reason about the outside world fail for an unglamorous reason: the material they reason over is incomplete, stale, or structurally unusable. This note argues that web context is a distinct infrastructure layer with its own quality properties, and that treating it as a solved detail of prompt engineering produces systems that look correct and are not.

Key questions

What specifically fails when an AI product is given raw web pages?
Which quality properties of source material most affect downstream output?
Why does this failure mode appear late in a product's life rather than early?
What would a mature web-context layer look like in practice?

How this note was developed

— Structured reading of public documentation from retrieval, crawling, and agent frameworks.
— Manual inspection of a varied set of public pages — documentation, pricing, listings, news — for the properties that make them hard to use.
— Construction of small prototype pipelines to observe where quality degrades.
— This is an independent analytical note. It contains no proprietary dataset, survey, or externally validated measurement.

Central arguments

01Page availability is not context availability

A successful HTTP response frequently contains no usable content: a shell awaiting client-side rendering, a consent interstitial, or a soft 404 returning 200. Pipelines that check status codes rather than content report high success rates while delivering nothing. The first quality gate should ask whether the retained text answers anything, not whether the request completed.

02Noise is not neutral

Navigation, footers, cookie notices, and related-article rails do not merely waste tokens. They add plausible text that competes with the content during retrieval, so a query about pricing can surface a footer link list from twelve different pages. Removing boilerplate improves retrieval precision before any ranking work happens.

03Freshness is a product property, not a cache setting

The acceptable age of a source depends on the claim being made. A definition can be a year old; a price cannot be a week old. Systems that apply one retention policy to all content will be either wastefully aggressive or quietly wrong, and the wrongness is invisible to the user.

04Traceability changes what a product is allowed to say

When every claim carries the URL and retrieval time it came from, the product can distinguish supported statements from inferred ones. Without provenance there is no mechanism for that distinction, so the interface has to present everything with equal confidence — which is the behaviour users later describe as hallucination.

05The failure appears late because early evaluation is generous

Demos use well-structured pages that the team chose. Production uses pages the user chose. The gap between those two populations is where the web context problem lives, and it is only visible if evaluation samples the messy end of the distribution deliberately.

Limitations

— The observations here come from prototype work and manual inspection, not from a controlled study.
— Page populations differ enormously by sector; conclusions drawn from documentation sites may not hold for listings or news.
— Model behaviour changes over time, so any claim about how models handle noisy context is provisional.

web contextAI infrastructureretrieval quality