Research note · Research note 06 — current edition
A Taxonomy of Web-Data Failure Modes
Written by Aaron Grainger
Independent Content Strategist & Product-Marketing Writer · Published Aug 19, 2025
- Primary audience
- Product managers building AI features
- Also useful for
- AI engineers and technical founders
- Tone
- Authoritative
- Reading time
- 2 min
- Published
- Aug 19, 2025
Abstract
Debugging a web-data pipeline is easier when failures have names. This note proposes a working taxonomy across five layers — access, retrieval, normalization, extraction, and interpretation — and notes which layer each symptom usually belongs to.
Key questions
- Where do the symptoms of a bad output actually originate?
- Which failures are detectable automatically and which require review?
- What is the cheapest assertion for each class?
How this note was developed
- — Collection of observed failures during prototype construction.
- — Grouping by originating layer rather than by symptom.
- — Analytical framework; no frequency data is presented.
Central arguments
01Access failures: blocked, throttled, gated, or geo-restricted
Usually loud, sometimes disguised as a 200 with a challenge page. Cheapest assertion: detect known interstitial patterns and content length far below the page-type norm.
02Retrieval failures: empty shells, partial renders, soft 404s
The request succeeded and the content did not arrive. Assertion: require a minimum of meaningful text and the presence of an expected structural element such as a heading.
03Normalization failures: over-stripping and under-stripping
Tables and code blocks dropped, or navigation retained. Assertion: compare retained element counts against the page-type profile and flag outliers for review.
04Extraction failures: wrong field, plausible invention, unit confusion
The most dangerous class, because output looks valid. Assertion: schema validation plus a required source excerpt for each extracted value.
05Interpretation failures: correct data, wrong conclusion
A stale page treated as current, a regional price treated as global, one vendor's claim treated as fact. No automated assertion suffices; this layer needs review policy and explicit provenance display.
Limitations
- — The taxonomy is a working tool, not an exhaustive or validated classification.
- — Layer boundaries blur — a soft 404 can equally be described as an access or retrieval failure.
Related reading
Related content
Research note · 2 min
A Framework for Evaluating Web-Enabled AI Workflows
Evaluating a web-enabled workflow with a single accuracy number hides where it is actually weak. This note proposes a five-dimension evaluation frame — coverage, freshness, fidelit
Documentation · 2 min
Technical Content Quality Standard
The editorial standard used for guides, documentation, and templates on this site — written to be copied and adapted.
Research note · 2 min
The Web Context Problem
Most AI products that reason about the outside world fail for an unglamorous reason: the material they reason over is incomplete, stale, or structurally unusable. This note argues
Research note · 2 min
The Anatomy of a Source-Linked AI Answer
A source-linked answer is not an answer with links appended. This note breaks a well-formed answer into its parts — claim, support, provenance, confidence, and refusal — and argues
Version history
Current: 1.0 · Published
- 1.0Aug 19, 2025First published.
Was this useful?
Sourceframe is an independent product concept created for research, product-design, and technical-content exploration. It is not an operating company, and nothing here describes a live commercial service. All examples, schemas, and code are illustrative unless a page says otherwise. No client data, customer outcomes, performance results, or partnerships are described anywhere on this site.