Documentation · Documentation version 1.0

Source Metadata Patterns

What to record about a retrieved page so a claim can be checked months later — and what not to store.

Independent Content Strategist & Product-Marketing Writer · Published Jan 13, 2026

Primary audience: AI application developers
Also useful for: AI engineers and technical founders
Tone: Authoritative
Reading time: 2 min
Published: Jan 13, 2026

In short

Source metadata is the record of where content came from and when. A workable minimum is the requested URL, the canonical URL, the retrieval timestamp, the content hash, the access method, and the excerpt supporting each claim. Store enough to re-check a claim; store as little third-party content as you can get away with.

On this page

The minimum record
Claim-level attribution
What not to store
Displaying provenance

The minimum record

Illustrative — retrieval recordjson · 13 lines

{  "requested_url": "https://example.com/docs/limits?utm_source=chat",  "canonical_url": "https://example.com/docs/limits",  "retrieved_at": "2026-09-14T09:12:04Z",  "access_method": "fetch",           // fetch | render | api | cache  "http_status": 200,  "content_hash": "sha256:9f2c...",  "content_chars": 5120,  "title": "Rate limits",  "published_hint": "2026-08-02",      // from the page, may be absent  "scope_rule": "docs.example.com/docs/*",  "schema_version": "limits.v3"}

Each field earns its place. The requested URL explains how the agent got there; the canonical URL is what a citation should show. The access method explains cost and fragility. The hash turns “did this change?” into a comparison rather than a re-read.

Claim-level attribution

Page-level provenance is not enough for anything a user will read. Attach each claim to the retrieval record and to the sentence that supports it.

Illustrative — claim recordjson · 9 lines

{  "claim_id": "c_18",  "text": "The free tier allows 60 requests per minute.",  "retrieval_id": "r_204",  "excerpt": "Free: 60 requests/minute, burst 120.",  "excerpt_contains_value": true,  "confidence_signals": ["excerpt_verbatim", "single_source"],  "conflicts_with": []}

What not to store

Whole pages, beyond a short retention window needed for debugging.
Personal data encountered incidentally — it is not part of the claim.
Content behind authentication your product was not authorized to use.
Screenshots as a substitute for text, unless the page is genuinely visual.

Displaying provenance

Surface, and what it should show

Surface	Show	Do not show
End-user answer	Canonical URL, retrieval date	Internal ids, model names
Hover or expansion	Supporting excerpt	Full page dump
Support tooling	Full retrieval record, access method	Unredacted third-party content
Export or report	URL, date, excerpt per claim	Confidence scores presented as accuracy

Provenance review

Every displayed claim resolves to a URL and a date
Excerpts are stored verbatim and are checkable against the claim
Stale content is labelled with its age rather than served silently
Retention windows for raw content are defined and enforced
Conflicting sources are represented, not collapsed

Practical takeaway

Requested URL and canonical URL are different facts — keep both.
A content hash makes change detection cheap and storage small.
Excerpts, not whole pages, are the unit worth retaining.
Metadata proves origin, never correctness.