Retrieval-augmented generation is often introduced as a prompt pattern: search for relevant passages, place them in the model context and ask for an answer. That description is technically recognizable and operationally incomplete. In production, the prompt is the last stage of a governed data path.
The system must decide which sources are eligible, how documents are segmented, what metadata travels with them, whose permissions apply at query time, which passages are selected, how claims connect to evidence and what is recorded for later review. Those decisions determine whether the answer is useful, traceable and appropriate for the user.
Why the model is not the whole system
The NIST AI Risk Management Framework calls for systems and their components to be evaluated under conditions similar to deployment, documented and monitored in production. The NIST Generative AI Profile treats grounding and retrieval-augmented generation as controls that must still be governed, measured and reassessed, including when third-party resources change.
That matters because retrieval can fail while the language remains fluent. A system can retrieve the wrong version, omit the decisive passage, expose material the user should not see, or return evidence that does not support the claim. A model-quality dashboard alone will not reveal those failures.
Five design decisions for a governed RAG platform
1. Define source eligibility
Do not begin with everything that can be indexed. Begin with what may be used for a named purpose. Give every collection an owner, a business purpose, an authoritative status, a classification and a review date. Exclude drafts, duplicates and records with unclear rights unless the use case explicitly permits them.
2. Carry identity into retrieval
Authorization must be enforced before passages enter the model context. A user who cannot open the source record should not receive its content through a generated answer. Preserve source permissions during ingestion, test them at retrieval and decide how permission changes remove previously indexed material.
3. Preserve provenance and version
Every retrieved passage should retain a stable source identifier, version or effective date, collection, owner and classification. Citations are not decoration. They are the path a user, operator or reviewer follows to verify why the system made a claim and whether the source was current.
4. Evaluate retrieval separately from generation
An answer can be wrong because the right evidence was not retrieved, or because the model mishandled good evidence. Measure those layers separately. For a representative question set, first inspect whether the needed passages appear in the retrieved set. Then assess whether the answer is supported, complete and appropriately qualified.
5. Treat changes as release events
A new embedding model, chunking rule, reranker, source collection or access-control mapping can change behaviour without changing the user interface. Record those dependencies, define who approves them and rerun the relevant evaluation set before release. Monitor retrieval and citation failures after launch, not only response latency.
The minimum release evidence
| Evidence | What it proves |
|---|---|
| Approved collection register with owners, purpose, classification and review date | The system retrieves from authorized, accountable sources. |
| Access-control tests for ordinary, privileged and revoked users | Source permissions survive indexing and retrieval. |
| Representative question set with expected source passages | Retrieval is measured against the use case, not a generic benchmark. |
| Answer review showing claim-to-source support and known gaps | Generated statements are traceable and limitations are visible. |
| Versioned release record for models, index, prompts and configuration | A production result can be reconstructed and a change can be assessed. |
What leadership should ask
- Who can authorize a source collection for this use case?
- Can a user receive content they could not open in the source system?
- Can we reconstruct which source version supported a material answer?
- Do we know whether a failure came from retrieval or generation?
If those answers are unclear, the architecture is not finished. The missing work is in the data product around the model.
If the source, permission, evaluation or release path is unclear, the RAG Platform Architecture practice is designed to make that path explicit.
Sources and scope
Primary references: NIST AI RMF Core, especially Measure 2 and Measure 3; NIST AI 600-1, Generative AI Profile; and the NIST Research Data Framework for provenance and fit-for-purpose data concepts. This article provides general architecture guidance. Controls must be adapted to the information, users and decisions in scope.
Want one practical architecture note each month?