Field note
The Difference Between Finding Information and Using It
Retrieval is a search problem. Usability is a structure problem. They fail in different places.
Written by Aaron Grainger
Independent Content Strategist & Product-Marketing Writer · Published Jun 2, 2026
- Primary audience
- AI application developers
- Also useful for
- AI engineers and technical founders
- Tone
- Conversational
- Reading time
- 2 min
- Published
- Jun 2, 2026
The premise
Teams debug web-enabled AI features as if there is one pipeline. There are two. Finding the right page is a selection problem, solved with queries, scoping, and ranking. Turning that page into something a model can use is a structure problem, solved with normalization, boundaries, and extraction. Mixing them means every bad answer gets the same fix — a prompt edit — and only half of them improve.
One symptom, two causes
The answer cites the right company and states the wrong number. That single symptom has two very different explanations. Either the system read the archived pricing page from a different locale — a finding failure — or it read the correct page and the number lived in a table your normalizer dropped — a usability failure.
You cannot tell which from the output. You can tell instantly from the stored intermediate: the URL that was used, and the cleaned text that was passed on.
Diagnosing in two minutes
| Question | If no | If yes |
|---|---|---|
| Is the canonical URL the page a careful human would pick? | Finding failure: fix scoping, queries, or selection scoring | Continue |
| Does the cleaned text contain the fact, verbatim? | Usability failure: fix normalization, rendering, or chunk boundaries | Continue |
| Did the extracted field match the excerpt? | Extraction failure: tighten the schema and require excerpts | The remaining gap is generation |
Why this keeps happening
Because the prompt is the most editable part of the system. When an answer is wrong, changing a sentence in the prompt takes ten seconds and re-running the pipeline takes two minutes, so that is what gets changed. Over a few weeks you accumulate a long prompt full of scar tissue instructing the model not to make mistakes it was never making.
What to change this week
- 01Log the canonical URL and cleaned-content length for every retrieval.
- 02Add the excerpt-contains-the-value check; it catches invention without a human reviewer.
- 03Before editing a prompt, classify the failure as finding, usability, extraction, or generation. Write the class in the ticket.
Practical takeaway
- Separate “did we reach the right source?” from “could we use what we reached?”.
- The two failures look identical in the output and differ completely in the fix.
- Log the intermediate artefact, or you are guessing which half broke.
- Most “it hallucinated” incidents are usability failures on a correct page.
Related content
Technical guide · 6 min
How AI Agents Browse the Web Reliably
A practical guide to discovery, retrieval, clean context, structured extraction, and source traceability.
Foundational guide · 4 min
How to Turn a Website Into LLM-Ready Markdown
Why clean, structured text often matters more than raw HTML in AI workflows.
Foundational guide · 7 min
The Web Context Layer: A Practical Architecture for AI Products
The layer between a user question and a model call — source selection, retrieval, normalization, structured context, and provenance.
Field note · 2 min
When a Search Result Is Not Enough
A ranked list answers “where might this be?” Products usually need “what does it say, and as of when?”
Version history
Current: 1.0 · Published
- 1.0Jun 2, 2026First published.
Was this useful?
Sourceframe is an independent product concept created for research, product-design, and technical-content exploration. It is not an operating company, and nothing here describes a live commercial service. All examples, schemas, and code are illustrative unless a page says otherwise. No client data, customer outcomes, performance results, or partnerships are described anywhere on this site.