Retrieval-Augmented Generation (RAG)
retrieval-augmented-generation · Frontier safety
An AI system pattern in which a model's outputs are conditioned on external content retrieved at inference time from a knowledge source — combining the parametric knowledge of the model with the up-to-date or domain-specific knowledge of the retrieval index.
Definition and scope
Retrieval-augmented generation was formalised by Lewis et al. (2020, 'Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,' NeurIPS) and is now the dominant pattern for deploying LLMs against proprietary, current, or specialised knowledge. The architecture pattern: at inference time, the user query is used to retrieve k documents from an index (vector store, search engine, structured database); those documents are appended to the prompt context; the model generates an answer conditioned on both its parametric memory and the retrieved context. RAG is the substrate for most enterprise LLM deployments — legal assistants citing case law, customer-support agents citing product docs, medical-AI citing clinical guidelines. Governance relevance opens a distinct surface from pure-LLM outputs. (a) Provenance — retrieved content has its own source attribution that must flow into the output; this is the technical substrate for citation-verifiability requirements (EU AIA Art. 50 transparency for AI-generated content). (b) Hallucination mitigation — RAG reduces but does not eliminate hallucination, because the model may still misquote or compositionally fabricate from retrieved sources. (c) Indirect prompt injection — the retrieval corpus is a primary adversarial-input vector (Greshake et al. 2023); an attacker who can plant content in the retrievable index can hijack the model. (d) Downstream-misinformation risk — RAG systems that surface low-quality sources amplify them with authoritative voice. (e) IP + training-data overlap — RAG creates a deployment-time analogue of training-data attribution questions, since retrieved-and-paraphrased content may infringe copyright at use-time. NIST AI RMF GenAI Profile §2.7 'Value Chain and Component Integration' is the closest binding frame; EU AI Act Art. 53 GPAI obligations apply to the model but the retrieval-index layer is largely unregulated.
Used by these instruments
Related concepts
- Hallucination— Confidently-asserted but factually incorrect output produced by an AI model — including fabricated c
- Prompt Injection— An adversarial input technique in which untrusted content fed to an AI model (e.g., text on a webpag
- Training-Data Attribution— Technical methods that identify which training examples most influenced a specific AI model output,
- AI Supply Chain— The end-to-end pipeline of inputs, intermediate artefacts, and downstream applications by which an A
- In-Context Learning— The capacity of a foundation model to adapt its behaviour to a new task purely from examples provide
Appears in topic articles
Editorial note
When citing RAG in policy contexts, distinguish the model-layer governance surface (EU AIA Art. 53 model-card obligations) from the retrieval-index-layer governance surface (largely unregulated). The retrieval layer is where most enterprise deployments concentrate risk in 2025-2026 because it sees less regulatory scrutiny than the underlying model.
References
Take this further — sign up free
Save, compare, or get alerts when Retrieval-Augmented Generation (RAG) changes. Policy Window is the analyst workbench layered on top of this wiki — free for researchers, civil society, and verified policymakers.