Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Glossary

RAG: definition of retrieval-augmented generation

Domains

Artificial intelligence · Data

In brief

RAG (retrieval-augmented generation) is an architecture in which a language model writes its answer from passages retrieved beforehand from a document base. It allows answers to draw on a private, up-to-date corpus, with verifiable sources, without retraining the model.

Page contents

Definition of RAG

RAG stands for retrieval-augmented generation. It is an architecture in which a large language model (LLM) does not answer from memory: before writing, the system searches a document base for the passages relevant to the question and provides them to the model, with the instruction to stick to them. The definition of RAG can be summed up as follows: search first, write second, from identified sources.

The expression comes from a research paper published in 2020. Today it refers to an architectural pattern more than a single technique, with many variants depending on how sources are searched and assembled.

What RAG is for

A language model only knows its training data. It is unaware of your internal documents and of everything after that training. RAG meets three needs.

  • Basing answers on a private corpus: procedures, contracts, technical documentation, knowledge base.
  • Keeping answers up to date without retraining the model: updating the document index is enough.
  • Making answers verifiable: each statement can point to the passage that supports it.

The stages of a RAG system

It works in two phases.

Preparing the corpus, upstream.

  1. Collecting documents and extracting their text, including from PDF files, tables or web pages.
  2. Splitting into passages of controlled size, respecting the document’s structure as far as possible.
  3. Vectorisation: an embedding model converts each passage into a numerical vector that represents its meaning.
  4. Indexing the vectors and their metadata (source, date, access rights) in a vector database or a search engine.

Processing a question, on demand.

  1. Retrieval: the question is vectorised in turn, then the system extracts the closest passages. Hybrid search combines this semantic proximity with keyword search.
  2. Reranking: a second model orders the candidates and discards the least relevant.
  3. Augmentation: the selected passages are inserted into the request sent to the model, with the question and the instructions.
  4. Generation: the model writes the answer and cites its sources.

The limits of RAG

RAG reduces factual errors, it does not eliminate them. Its failures have identifiable causes.

  • Retrieval fails. If the right passage is not retrieved, the model answers beside the point or fills the gap. The quality of the search determines that of the answer.
  • The corpus is deficient. Contradictory versions, outdated documents, badly extracted tables: the system faithfully reproduces a poor-quality document collection.
  • Splitting breaks the meaning. A rule and its exception separated into two passages produce an answer that is wrong and yet sourced.
  • Access rights are forgotten. Without per-user filtering at search time, the system may reveal a document the requester is not entitled to read.
  • Synthesis questions are beyond it. “How many contracts contain this clause?” requires going through the whole corpus, not retrieving a few passages.

Evaluation therefore distinguishes two measures: the relevance of the retrieved passages, then the faithfulness of the answer to those passages.

RAG and fine-tuning: not to be confused

Fine-tuning continues a model’s training on selected examples, which modifies its parameters. RAG does not touch the model: it changes what the model is given to read.

RAG Fine-tuning
What changes The context provided to the model The model’s parameters
Main contribution Up-to-date, sourced knowledge A style, a format, a behaviour
Updating Reindex the documents Retrain the model
Traceability Sources cited in the answer No source returned

The two approaches can be combined. To bring in knowledge, RAG is generally the first choice. Fine-tuning meets another need: a business vocabulary, a strict output format, a repetitive task.

Finally, an AI agent can use retrieval as one tool among others, deciding for itself when to search and what to search for.

A concrete example: a support centre assistant

Consider a technical support centre whose advisers consult several thousand procedure sheets. A RAG-based assistant receives an adviser’s question, finds the few passages applicable to the equipment model concerned, then writes an answer that cites each sheet used.

The first trial often disappoints, for a reason unrelated to the model: two versions of the same procedure coexist in the collection, and the assistant cites the old one. The fix is documentary. Obsolete versions must be removed, sheets dated and the search filtered on the version in force.

To implement these architectures, see our page devoted to generative AI in the enterprise.

Insights

Further reading

All insights
AI in banking, one use case after another

AI in banking covers four families of use cases (customer relations, fraud prevention and compliance, credit decisions, internal operations) that require neither the same data, nor the same control, nor the same regulatory reading. Management Square proposes here criteria for choosing them and an order for tackling them, from uses that a person reviews to those that affect a customer's situation.

Read the analysis
Building the budget for an AI project, from cost items to the choice between build and buy

The budget for an AI project can be broken down into six items: framing, data, build, evaluation, operation and compliance, part of which is spent once and part of which recurs as long as the service exists. Management Square describes each item here by its nature and by what makes it vary, then what changes with the choice between building, adapting an existing model and buying a service.

Read the analysis
The difference between AI and generative AI: when to choose a predictive model, when to choose a generative model

The difference between AI and generative AI lies in what the system produces: AI refers to the whole field, including predictive models that estimate a value or classify a case, whereas generative AI, one of its branches, produces new content. Management Square compares the two families here and indicates in which situations each is the right choice.

Read the analysis

FAQ

Frequently asked questions

Another question? Ask us

What does RAG mean in artificial intelligence?

RAG stands for retrieval-augmented generation. The system first searches a document base for the relevant passages, then passes them to a language model which writes the answer based on them and cites its sources.

What is the difference between RAG and fine-tuning a model?

RAG does not modify the model: it provides it with documents to read at the time of the question. Fine-tuning continues the model's training and changes its parameters. The first brings up-to-date, sourced knowledge, the second adapts a style, a format or a behaviour.

Does RAG eliminate hallucinations?

No. RAG reduces factual errors by giving the model sources to follow, but it does not eliminate them. If the search retrieves the wrong passages, or if the corpus contains outdated or contradictory documents, the answer will be wrong while appearing sourced.

Do you need a vector database to do RAG?

Not necessarily. Search by semantic proximity relies on vectors, but many existing search engines and databases offer this function. Hybrid search, which combines vectors and keywords, often gives better results than vectors alone.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.