Definition of RAG
RAG stands for retrieval-augmented generation. It is an architecture in which a large language model (LLM) does not answer from memory: before writing, the system searches a document base for the passages relevant to the question and provides them to the model, with the instruction to stick to them. The definition of RAG can be summed up as follows: search first, write second, from identified sources.
The expression comes from a research paper published in 2020. Today it refers to an architectural pattern more than a single technique, with many variants depending on how sources are searched and assembled.
What RAG is for
A language model only knows its training data. It is unaware of your internal documents and of everything after that training. RAG meets three needs.
- Basing answers on a private corpus: procedures, contracts, technical documentation, knowledge base.
- Keeping answers up to date without retraining the model: updating the document index is enough.
- Making answers verifiable: each statement can point to the passage that supports it.
The stages of a RAG system
It works in two phases.
Preparing the corpus, upstream.
- Collecting documents and extracting their text, including from PDF files, tables or web pages.
- Splitting into passages of controlled size, respecting the document’s structure as far as possible.
- Vectorisation: an embedding model converts each passage into a numerical vector that represents its meaning.
- Indexing the vectors and their metadata (source, date, access rights) in a vector database or a search engine.
Processing a question, on demand.
- Retrieval: the question is vectorised in turn, then the system extracts the closest passages. Hybrid search combines this semantic proximity with keyword search.
- Reranking: a second model orders the candidates and discards the least relevant.
- Augmentation: the selected passages are inserted into the request sent to the model, with the question and the instructions.
- Generation: the model writes the answer and cites its sources.
The limits of RAG
RAG reduces factual errors, it does not eliminate them. Its failures have identifiable causes.
- Retrieval fails. If the right passage is not retrieved, the model answers beside the point or fills the gap. The quality of the search determines that of the answer.
- The corpus is deficient. Contradictory versions, outdated documents, badly extracted tables: the system faithfully reproduces a poor-quality document collection.
- Splitting breaks the meaning. A rule and its exception separated into two passages produce an answer that is wrong and yet sourced.
- Access rights are forgotten. Without per-user filtering at search time, the system may reveal a document the requester is not entitled to read.
- Synthesis questions are beyond it. “How many contracts contain this clause?” requires going through the whole corpus, not retrieving a few passages.
Evaluation therefore distinguishes two measures: the relevance of the retrieved passages, then the faithfulness of the answer to those passages.
RAG and fine-tuning: not to be confused
Fine-tuning continues a model’s training on selected examples, which modifies its parameters. RAG does not touch the model: it changes what the model is given to read.
| RAG | Fine-tuning | |
|---|---|---|
| What changes | The context provided to the model | The model’s parameters |
| Main contribution | Up-to-date, sourced knowledge | A style, a format, a behaviour |
| Updating | Reindex the documents | Retrain the model |
| Traceability | Sources cited in the answer | No source returned |
The two approaches can be combined. To bring in knowledge, RAG is generally the first choice. Fine-tuning meets another need: a business vocabulary, a strict output format, a repetitive task.
Finally, an AI agent can use retrieval as one tool among others, deciding for itself when to search and what to search for.
A concrete example: a support centre assistant
Consider a technical support centre whose advisers consult several thousand procedure sheets. A RAG-based assistant receives an adviser’s question, finds the few passages applicable to the equipment model concerned, then writes an answer that cites each sheet used.
The first trial often disappoints, for a reason unrelated to the model: two versions of the same procedure coexist in the collection, and the assistant cites the old one. The fix is documentary. Obsolete versions must be removed, sheets dated and the search filtered on the version in force.
To implement these architectures, see our page devoted to generative AI in the enterprise.