A retrieval-augmented generation demo can be built quickly: split documents into chunks, create embeddings, retrieve the nearest results, and place them in a prompt. That proves the mechanics work. It does not prove the system can answer a business question safely or consistently.
Production RAG needs an evidence architecture around retrieval.
Ingest with identity and provenance
Every indexed item should retain its source, version, owner, timestamp, and access policy. A chunk without provenance is difficult to review and impossible to retire cleanly when the underlying document changes.
Stable source identifiers also make re-indexing safer. The ingestion process can replace content from a known document instead of accumulating duplicate versions in the vector store.
Retrieve within the user's permissions
Similarity is not authorization. The retrieval layer must apply organization, user, role, and sensitivity rules before context reaches the model. Filtering only after generation is too late because restricted content may already have influenced the answer.
For multi-tenant applications, tenant isolation should be enforced in the data access layer and covered by tests. A prompt instruction is not a security boundary.
Treat retrieved text as evidence
Retrieved passages should support specific claims in the answer. Keep the relationship between a claim and its evidence so the interface can show why the system reached a conclusion.
This is stronger than adding a list of links at the end. A reviewer should be able to inspect the passage that supports an important statement and see whether the statement goes beyond the source.
Handle conflicts and missing support
Business knowledge is rarely consistent. Two policies may disagree, a newer document may supersede an older one, or a source may not address the question at all.
The system should preserve dates and source priority, surface meaningful conflicts, and allow an explicit "insufficient evidence" result. Forcing the model to produce a complete answer turns missing information into confident fabrication.
Evaluate the whole retrieval path
Offline evaluation should include realistic questions, expected evidence, permission scenarios, and cases that should not be answered. Useful measurements include retrieval recall, ranking quality, citation correctness, answer support, and abstention behavior.
Monitor production separately. Query patterns, empty retrievals, source freshness, latency, user corrections, and review outcomes reveal problems that a static benchmark may miss.
Keep the model replaceable
The ingestion pipeline, authorization policy, retrieval logic, evidence representation, and evaluation suite should not depend on one model provider. Models will change faster than the knowledge and compliance requirements around them.
A clean boundary makes it possible to test a new embedding model, reranker, or generator without redesigning the whole application.
Build for correction
When a reviewer identifies a bad answer, the team needs to know whether the failure came from missing source material, incorrect permissions, poor retrieval, an unsupported synthesis, or stale knowledge. Evidence lineage turns feedback into a specific engineering action.
That is the difference between a RAG feature and a production knowledge system. Retrieval supplies context. Provenance, policy, evaluation, and correction make that context dependable.