Definition of the LLM
An LLM, for large language model, is a neural network trained on very large volumes of text to predict the continuation of a sequence. The definition of the LLM lies in this mechanism. From an input text, the model calculates which fragment is most likely to come next, adds it, then starts again. The answers, summaries, translations or code it produces result from this repeated prediction.
Current LLMs are based on the so-called transformer architecture, presented in 2017, whose attention mechanism makes it possible to take into account the relationships between words that are far apart in a text. The qualifier “large” refers to the number of parameters, that is, the numerical values adjusted during training.
What an LLM is for
A single model covers tasks that previously required separate tools.
- Writing, rephrasing, summarising and translating.
- Classifying texts and extracting information from unstructured documents.
- Answering questions about a corpus provided to it.
- Writing, explaining and correcting code.
- Driving tools, when integrated into an AI agent.
An LLM is neither a database nor a calculator. It does not consult a stock of facts: it produces plausible text. This property explains both its versatility and its errors.
Tokens and context window
An LLM does not read words but tokens: fragments of text that correspond to a short word, a piece of a word or a punctuation mark. The token is the unit for everything: the length of what the model can read, the length of what it can write, and the billing of model access services, most often counted in input and output tokens.
The context window is the maximum number of tokens the model takes into account at once: instructions, documents provided, conversation history and the answer in progress. What is not in it does not exist for the model. Two practical consequences follow. A model has no memory from one exchange to the next, unless the application provides the history again. And a large document collection does not fit in the window, which leads to selecting the useful passages according to the principle of retrieval-augmented generation (RAG).
The hallucinations of an LLM
A hallucination is a wrong answer stated with the same confidence as a correct one: an invented reference, an incorrect figure, a clause that does not appear in the contract. The phenomenon stems from the very way the model works, as it produces the most plausible text and not verified text.
It can be reduced but not eliminated. The levers are known: providing sources to the model, requiring it to cite the passage it relies on, narrowing the task, automatically checking outputs, and keeping human review wherever an error is costly.
Open models and proprietary models
| Proprietary model | Open-weight model | |
|---|---|---|
| Access | Programming interface of the vendor or of a cloud provider | Downloadable parameters, hosted by you or by a third party |
| Control | Dependence on changes, pricing and withdrawal of versions | Choice of version and of where processing takes place |
| Operations | Provided by the vendor | Compute infrastructure and skills to be planned |
| Adaptation | Within the limits set by the vendor | Fine-tuning possible on your data |
The word “open” calls for case-by-case verification. Publishing the parameters says nothing about the training data, and licences vary: some restrict commercial use. The choice is made on the sensitivity of the data processed, the volume of calls, the level of quality required and the skills available to operate the model.
LLM, generative AI, assistant: not to be confused
Generative AI is the family of systems that produce content: text, image, sound, video. The LLM is its textual branch, even though many models now also process images or sound. A conversational assistant is an application built around an LLM, with an interface, instructions and sometimes tools. The model is the engine, the assistant is the product.
From a legal standpoint, the European regulation on AI uses another category, that of general-purpose AI models, which large LLMs fall under. It is described in our entry on the AI Act.
A concrete example: contract analysis
Take a legal department that wants to identify certain clauses in a collection of long contracts. The preceding notions become design questions. Does each contract fit in the context window, or does it need to be split? Does the cost, proportional to the number of tokens, remain acceptable once multiplied by the number of contracts?
The risk of hallucination is handled by design: the model must quote the selected passage word for word, and an automatic check verifies that the quotation does appear in the document. Finally, the confidentiality of the contracts guides the choice between a proprietary model backed by contractual guarantees and an open-weight model hosted in-house.
For choosing and integrating these models, see our page dedicated to generative AI in the enterprise.