Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Glossary

LLMOps: definition, components and differences with MLOps

Domains

Data · Artificial intelligence

In brief

LLMOps (large language model operations) refers to the practices and tools used to put into production, evaluate, monitor and evolve an application built on a large language model. A term of common usage, whose scope varies from one source to another, LLMOps extends MLOps and covers in particular prompt management, evaluation, monitoring in production and cost control.

Page contents

Definition of LLMOps: a term of common usage

LLMOps, for large language model operations, refers to the practices and tools used to put into production, evaluate, monitor and evolve an application built on a large language model (LLM): an assistant, a document search system, an agent.

The word is formed on the model of MLOps. It is one of common usage, and its scope varies from one source to another. This entry takes the meaning of an extension of MLOps, organised around four subjects: prompts, evaluation, monitoring in production and costs.

What makes operating an LLM-based application distinctive

The behaviour of such an application does not depend on its code alone. It depends on the prompt, the version of the model called, the generation parameters and, where there are any, the documents provided to the model. A change to one of these elements can change the answers without a single line of code having moved.

Three other traits are added. The output is free text, which can vary from one call to another for the same request. When the model is provided by a third party, its versions evolve according to a schedule the organisation does not set. Finally, when access to the model is billed on usage, the cost grows with the volume of text read and produced, counted in tokens.

Prompts, evaluation, monitoring, costs: the four components

Component What it organises
Prompt management Versioning of instructions and of call configuration (model version, parameters), their review and their release like code
Evaluation A set of reference cases, with expected results fixed in advance, submitted to the application before each change of prompt, model or corpus, then a sample of real exchanges reviewed after release
Monitoring in production Traces of each exchange: request, passages provided, answer, latency, number of tokens, user feedback, guardrail triggered
Cost control Measuring cost per request and per use, and the levers that reduce it: context length, number of calls per answer, choice of model according to the task

A trace contains the text of the request, which may include personal data: what it retains, and for how long, must be decided before release.

LLMOps, MLOps, prompt engineering, RAG: not to be confused

MLOps concerns models that the organisation trains, and our entry on MLOps describes their lifecycle. LLMOps takes up its principles (versioning, testing, monitoring) and shifts their object. Training leaves the cycle when the model is used as is. The model registry becomes a registry of prompts and configurations. Measuring performance on labelled data gives way to assessing free text.

The two meet when a language model is fine-tuned on examples specific to the organisation: a trained artefact returns to the pipeline. Our article on LLM fine-tuning sets out this case.

Prompt engineering is the art of writing an instruction. LLMOps deals with what surrounds it: version, test, deployment, monitoring. Finally, RAG is an application architecture, whose document index becomes one more object to version and evaluate.

Example: the announced withdrawal of a model version

Take an assistant that answers employees’ questions on leave rules, based on the company’s agreements. The model provider announces the withdrawal of the version in use. The team replays the set of reference cases on the next version. The answers remain accurate, but the assistant now answers out-of-scope questions it previously refused.

The prompt is corrected, recorded under a new version, then re-evaluated. The cost per request is compared between the two configurations. The new version is opened to a portion of users, and the traces are reviewed before general rollout. The old configuration remains available until the withdrawal.

Setting up this pipeline is the subject of our page on MLOps and LLMOps. The design of the assistant itself (task, corpus, architecture) falls under generative AI in the enterprise.

FAQ

Frequently asked questions

Another question? Ask us

What does LLMOps mean?

LLMOps combines LLM (large language model) and operations. The word refers to the practices and tools that govern the operation of an application built on a language model: prompt management, evaluation, monitoring in production and cost control. It is a term of common usage, formed on the model of MLOps, whose scope varies from one source to another.

What is the difference between LLMOps and MLOps?

MLOps organises the lifecycle of a model that the organisation trains: training data, training, statistical validation, retraining. LLMOps applies when the application calls an already trained language model, which may be provided by a third party. The work then shifts to prompts, call configuration, evaluation of free text and usage-based cost.

What is prompt management?

Prompt management consists of treating the instructions sent to the language model as versioned artefacts, in the same way as code. Each version is recorded with its accompanying configuration (model version, generation parameters) and replays the evaluation set before being put into service. A return to the previous version thus remains possible.

Is an LLMOps approach useful when the model is provided by a third party?

Yes. When the language model is provided by a third party, the organisation controls neither its training nor its release schedule, but it remains responsible for the behaviour of its application. Prompts, documents provided to the model, evaluation, traces and cost per request remain its responsibility, and it is these that LLMOps organises.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.