Definition of MLOps
MLOps, a contraction of machine learning and operations, refers to the set of practices, roles and tools that make it possible to put machine learning models into production, then keep them there reliably.
The definition of MLOps extends that of DevOps: automate, version, test, monitor. It adds what is specific to models. Their behaviour depends on data as much as on code, and it degrades over time even when nothing has been changed.
What MLOps is for
A model that gives good results in an experimentation notebook is not yet a service. Between the two lie questions that experimentation did not have to address. Can this model be rebuilt identically? Who deploys it, and how do you roll back? How do you know whether it is making more mistakes than on day one? What happens if its author changes role?
MLOps brings industrial answers to these questions. It serves four objectives: reproducibility of training runs, speed and safety of production releases, control of quality over time, and traceability. The latter means being able to say which data, which code and which model version produced a given prediction.
The life cycle of a model
- Framing. Definition of the need, the measure of success and the available data.
- Data preparation. Collection, cleaning, construction of input features.
- Experimentation. Training of several candidates, comparison of results.
- Validation. Tests of performance, robustness and fairness on data the model has not seen.
- Deployment. Real-time serving or batch execution, often progressively, on a fraction of the traffic.
- Monitoring. Technical monitoring of the service, monitoring of input data and monitoring of prediction accuracy.
- Retraining or retirement. Updating the model when its performance drops, decommissioning it when it is no longer useful.
This cycle is a loop. The monitoring stage feeds retraining, and a model in service regularly goes back through validation.
The building blocks of an MLOps pipeline
- Experiment tracking records the parameters, data and results of each training run.
- The model registry stores validated versions, their status and their history.
- The feature store makes the same features available for training and for production.
- Orchestrated pipelines chain preparation, training, validation and deployment without manual intervention.
- Monitoring triggers alerts in the event of data drift or a drop in performance.
Not all of them are needed from the outset. The level of automation is set according to the number of models in service and the frequency of their updates. Training data can come from the organisation’s data platform, for example a lakehouse.
MLOps, DevOps, LLMOps: not to be confused
DevOps deals with code. A given version of the code always produces the same behaviour, and tests verify it before the production release.
MLOps deals with three objects: the code, the data and the trained model. Each one is versioned. Tests are partly statistical, since a performance has to be judged rather than a binary correctness. Monitoring also covers drift, that is, the growing gap between training data and production data. Continuous deployment is paired with continuous training.
LLMOps transposes these principles to applications built on a large language model (LLM). When the model is provided by a third party, the organisation does not train it. The objects to version become the prompts, the retrieval configuration and the evaluation datasets. Evaluation is trickier, because the output is free text. Added to this are tracking the cost per token and managing version changes decided by the model provider.
A concrete example: drift in a forecasting model
Take a retailer that forecasts its sales by store using a model trained on its history. The retailer changes its pricing policy and renews part of its product range. The model keeps running without any technical error, but its forecasts gradually move away from actual sales.
Without MLOps, it is the supply planners who notice, late, when they see stock-outs and overstocks. With an MLOps pipeline, monitoring continuously compares forecasts with sales and detects the change in input data. An alert triggers retraining on recent data. The new model is compared with the old one before being promoted, and the old one remains available for a rollback.
For the industrialisation of your models, see our page on MLOps and putting models into production.