Checklist / RAG and knowledge systems / Intermediate
RAG Content Readiness Checklist
A pre-ingest review of whether a content set is worth indexing yet. Directional, not a formal audit.
For Platform and documentation teams
01Discovery
- A sitemap or equivalent URL list exists and resolves
- Crawl scope is defined by path rules, not by guesswork
- Versioned or localised duplicates are identified
- Canonical tags are present and respected by the crawl
02Content
- Pages carry a single clear H1 and a sane heading hierarchy
- Code blocks and tables survive normalization intact
- Navigation, footers, and banners are removed from retained text
- Pages that are pure navigation are excluded from the index
03Metadata
- Each document records title, canonical URL, and section path
- Product or version is captured where the content is version-specific
- A last-retrieved timestamp is stored with every document
04Maintenance
- A recrawl cadence is agreed and owned by someone
- Removed pages are deleted from the index, not left orphaned
- A small retrieval test set exists and is run after each recrawl
ragdocumentationcrawling