A sovereign RAG architecture is a retrieval-augmented generation set-up in which each component runs in an environment the organisation has verified: where processing takes place, which law the provider is subject to, who operates the service and holds the keys. “Sovereign” does not refer to any label here. The word sums up these checks, which our page sovereign cloud develops.
The decision is taken component by component, because a RAG has at least five and each receives data in clear: the corpus preparation pipeline, the embedding model, the vector index, the generation model and the logs. Placing the index in a protected environment does not protect the retrieved passages, which are sent to the generation model with every question. The level of requirement is that of the class of data in the corpus; a corpus with no sensitive data does not call for these precautions.
The components of a RAG and the data that passes through them
| Component | Data received in clear | Point to check |
|---|---|---|
| Preparation pipeline (extraction, chunking) | Every document in the corpus, in full | The text extraction or character recognition services called, and where they run |
| Embedding model | Every passage at indexing, every question in use | The service called, separate from the generation model, and its provider |
| Vector index | The vectors, the text of the passages, the metadata and the access rights | Managed service or database run by your teams, backups and copies included |
| Generation model | The instruction, the question and the retrieved passages | Place of inference, retention of requests, reuse for training |
| Logs and traces | Questions, passages, answers, user identity | Tracing and evaluation tool, retention period, support access |
A reranking model, if used, receives the question and the candidate passages: it is treated like the generation model.
The generation model receives the passages chosen by retrieval
With every question, this component receives a targeted extract of the corpus, together with the user’s request. Three modes of execution can be distinguished.
In the first, the application calls a model provider’s interface. The contract and the service documentation state which entity is committed, where inference takes place, how long requests are retained, including for abuse detection, and whether they are used for training.
In the second, a cloud provider serves the model in one of its regions. The check concerns the inference service itself: an infrastructure that meets the requirements grid says nothing about the same provider’s managed services.
In the third, your teams host an open-weight model, on internal infrastructure or on an infrastructure service that complies with the grid. A model provider’s terms no longer apply to requests; the model’s licence still has to be read. Compute capacity, operations and version changes fall to you.
The quality of answers is measured on your reference question set, following the approach described in our page generative AI in the enterprise, and the gap observed between two modes enters into the decision on the same footing as the constraint.
The entire corpus passes through the embedding model and the index
The embedding model calls for the same examination as the generation model. At indexing, it receives every passage of the corpus; in use, every question. If it is called as a service outside the perimeter, the entire corpus has left the perimeter once, even if generation is then done internally.
This choice also affects reversibility. Vectors produced by two different embedding models cannot be compared: changing model means vectorising the whole corpus again. The source corpus and its preparation pipeline therefore remain under your control, so that the index can be rebuilt elsewhere.
The index, for its part, is placed where the original documents would be allowed to reside. It can be a dedicated vector database, a search engine such as OpenSearch or an extension of a relational database, such as pgvector for PostgreSQL. When the corpus and the index rely on the organisation’s data platform, its access and retention rules apply to them; designing this foundation falls under our data platform offering.
The logs copy what the rest of the architecture protects
For diagnosis and evaluation, a RAG application keeps the questions, the passages passed on and the answers. These traces therefore contain what the other components handle, gathered in a single place.
Several destinations need locating: the tracing tool, if it is used as a service; the model to which an automatic evaluation set-up submits samples of answers; the answer cache; the logs kept by the model provider itself.
A question linked to an identified user is personal data, and it reveals information about the case they are handling. The GDPR, Regulation (EU) 2016/679, applies to these logs, whose retention period is set at design time. Its Chapter V governs transfers of personal data to third countries.
The location and qualification criteria of a sovereign RAG architecture
For each row of the table, the checks are carried out in this order.
- The class of data passing through the component, according to the organisation’s classification. The component inherits the most demanding class it receives.
- Location, extended to backups, copies and support data.
- The provider operating the service, the law it is subject to and its subcontracting chain. For personal data, Article 28(2) of the GDPR prohibits the processor from engaging another processor without the prior specific or general written authorisation of the controller.
- Qualification, if your grid requires it. SecNumCloud is ANSSI’s qualification framework for cloud service providers, and the qualification applies to a specific service. It is checked for each managed service used, against the list published by ANSSI.
- Obligations specific to the corpus. Personal health data brings into play the Health Data Host certification (Hébergeur de données de santé, Article L. 1111-8 of the French Public Health Code), which is distinct from the SecNumCloud qualification.
When the sovereignty constraint is not justified
The constraint has a price: a narrower choice of models, compute capacity to reserve, operations to provide. If measured quality suffers, users may turn to consumer tools, and the risk the architecture was meant to contain moves out of your sight.
It is not justified for a corpus that is already public, such as product documentation or regulatory texts. Only users’ questions may then remain sensitive, which brings the examination back to the logs. Nor is it justified for data that your classification allows on a standard service backed by contractual commitments, or for a pilot run on public or fictitious documents, which tests the pipeline before any investment in hosting.
Between these situations, a corpus can be split according to the sensitivity of its documents. Only the sensitive part bears the constraint, and the organisation runs two pipelines instead of one.
For a project already under way, the work begins with a record of the service actually called by each component and of the class of data it receives. A gap with the organisation’s grid is resolved by moving the component, or by removing from the corpus the documents that impose the constraint.