Research note · Research note 07 — current edition
The Difference Between Finding Information and Using It
Written by Aaron Grainger
Independent Content Strategist & Product-Marketing Writer · Published Dec 3, 2024
- Primary audience
- Product managers building AI features
- Also useful for
- AI engineers and technical founders
- Tone
- Authoritative
- Reading time
- 2 min
- Published
- Dec 3, 2024
Abstract
Search solves location. Most AI products fail at the step after location: turning a set of plausible pages into material a system can act on. This note separates the two problems and argues that conflating them is why 'add search' rarely fixes an unreliable product.
Key questions
- What does a search result actually give a system?
- Which work remains between a relevant URL and a usable fact?
- Why does adding search sometimes make output quality worse?
How this note was developed
- — Walkthrough of the steps between a query and a usable claim.
- — Comparison of what each step can and cannot guarantee.
- — Independent analysis; no measurement of search quality is offered.
Central arguments
01A result set is a hypothesis, not evidence
Ranking predicts relevance to a query, not the presence of a specific fact. A system that treats the top result as an answer has outsourced its evidentiary standard to a ranking function built for a different purpose.
02Adding search can lower quality by raising volume
More candidate pages means more opportunities to retrieve noise, and models are not reliably better at ignoring confidently written irrelevant text than at ignoring nothing at all. Selection needs to tighten as recall widens.
03Usability is a property of the page, not the query
Two equally relevant pages can differ completely in whether the needed fact is extractable. A pipeline should be able to reject a relevant page as unusable and move on rather than forcing an extraction.
04The gap is where product differentiation lives
Search quality is broadly available. What a product does between finding and using — selection, normalization, extraction, verification — is where reliability differences accumulate.
Limitations
- — Search behaviour differs across providers and query types; general claims are approximate.
- — No comparative evaluation of search products is made or implied.
Related reading
Related content
Field note · 3 min
Your AI Agent Does Not Need “the Internet.” It Needs a Retrieval Plan.
“Give the agent web access” is a capability statement pretending to be a requirement.
Field note · 2 min
When a Search Result Is Not Enough
A ranked list answers “where might this be?” Products usually need “what does it say, and as of when?”
Research note · 2 min
The Web Context Problem
Most AI products that reason about the outside world fail for an unglamorous reason: the material they reason over is incomplete, stale, or structurally unusable. This note argues
Research note · 2 min
The Anatomy of a Source-Linked AI Answer
A source-linked answer is not an answer with links appended. This note breaks a well-formed answer into its parts — claim, support, provenance, confidence, and refusal — and argues
Version history
Current: 1.0 · Published
- 1.0Dec 3, 2024First published.
Was this useful?
Sourceframe is an independent product concept created for research, product-design, and technical-content exploration. It is not an operating company, and nothing here describes a live commercial service. All examples, schemas, and code are illustrative unless a page says otherwise. No client data, customer outcomes, performance results, or partnerships are described anywhere on this site.