The budget for an AI project is spread over two horizons: that of the project, which has an end date, and that of the service, which runs until it is withdrawn. The first carries framing and build. The second carries model usage, re-evaluation of the answers and updating of the data.
This budget can be broken down into six items: framing, data, build, evaluation, operation and compliance. Framing and build are mostly spent once. Operation recurs every period. Data, evaluation and compliance have a share of each kind. The weight of each item depends on the option chosen: building, adapting an existing model or buying a service. We describe these items by their nature and by what makes them vary, without amounts: a figure is established project by project, on the basis of written assumptions. Measuring the gains is the subject of our article on the ROI of artificial intelligence.
The items in an AI project budget and what makes them vary
| Item | What it funds | One-off or recurring | What makes it vary |
|---|---|---|---|
| Framing | Choice of use case, record of the current process, success and stopping criteria | One-off, repeated at each extension of scope | Number of departments involved, degree to which the process is described |
| Data | Access to sources, preparation, rights, updating the corpus or example sets | One-off for preparation, recurring for updates | State of the sources, number of owners, presence of personal data |
| Build | Development, integration with applications, access security | One-off | Number of connected applications, sourcing option |
| Evaluation | Reference case set, business experts’ time, replayed campaigns | One-off for building the set, recurring at each change | Frequency of changes of model, prompt or corpus, share of human judgement |
| Operation | Model usage or compute capacity, monitoring, support, maintenance | Recurring | Volume of requests, length of texts processed, number of calls per answer |
| Compliance | System classification, assessments, documentation, human oversight | One-off for the examination, recurring for keeping it up to date | System risk level, organisation’s role, personal data |
Business teams’ time runs across several lines: the experts who build the reference set, the managers who judge the answers, the users who get to grips with the tool. If it is not counted on any line, it is taken from day-to-day activity without having been decided.
The compliance item is costed once the system has been classified. Regulation (EU) 2024/1689 attaches its obligations to the organisation’s role and to the system’s risk level. This classification is examined at the framing stage, with your lawyers, within the framework described in our page on AI governance.
The one-off and recurring shares of spending
The one-off share takes the form of a project budget: a start, an end, a deliverable. The recurring share starts at go-live and is renewed every financial year, until the service is withdrawn.
For a service based on a language model billed by usage, the operation line follows three quantities: the number of requests, the volume of text read and produced per request, counted in tokens, and the number of model calls needed for one answer. These quantities are taken from operating traces, according to LLMOps practices.
Let us take a calculation assumption, which is not a measurement. An assistant receives 1,500 requests per working day and sends the model 4,000 tokens per request, prompt and passages included: the volume read reaches 6 million tokens per day. If a second verification pass, of the same volume, is added to each answer, this volume doubles without a single extra user being served. The operation line is therefore written as a formula, in which each assumption remains visible and is replaced by a measurement from the pilot onwards.
When the model is hosted by the organisation, the formula changes: spending depends on the reserved compute capacity and on the team that operates it, and it runs even when requests are few.
Some recurring spending follows an event rather than the calendar. The provider withdraws a model version: the reference set is replayed and the prompt reworked. A documentary source is overhauled: the corpus is prepared again. The purpose widens: the classification is re-examined. A contingency entered in the budget avoids treating each of these events as an overrun.
It remains to designate the department that will carry the recurring line once the project has closed. Tracking this usage-based spending borrows the practices of our FinOps page: an owner for each cost, and a unit cost tracked from one period to the next.
Building, adapting an existing model or buying a service
The choice between these options is made use case by use case, according to the criteria of our AI strategy page. What follows describes only how each one shifts the items.
Buying a service, whether an AI feature in software already in place or an off-the-shelf assistant, reduces the build to configuration and integration. Operation takes the form of a subscription or usage-based billing, set by the contract. Three items remain the buyer’s responsibility: preparing the data the service will read, evaluation on its own cases, and the compliance attached to its role. One line is added, that of exit: recovering its data and configuration the day the service is replaced.
Adapting an existing model covers several degrees. A carefully crafted prompt weighs little on the build. Document retrieval, on the RAG principle, enlarges the data item: the corpus is prepared, then kept up to date. Fine-tuning the model adds examples to produce, a training run and an artefact to retrain when the base model changes, as detailed in our article on fine-tuning an LLM.
Building mobilises every item, and adds skills to retain over time as well as a production release and monitoring pipeline, the subject of our MLOps and LLMOps page.
The trade-off we propose is made on full cost. An option that is light to build can become heavy to operate as volume grows, since usage-based billing follows requests. An option that is heavy to build can cost little per request once the capacity is installed, provided that capacity is used. The comparison therefore requires a volume assumption at several horizons, written down before the choice.
A budget requested in tranches, backed by assumptions
We propose a method: presenting the budget in tranches, each accompanied by a criterion for continuing and a criterion for stopping.
The first tranche funds framing and a proof on real data: a data scope, a reference set, a measured prototype. It produces the volume and quality measurements that the assumptions lacked. The second funds the build, and is committed only in the light of these measurements. Then comes the operation line, presented with its formula, its contingency and the department that carries it.
Certain situations justify stopping at the first tranche: no department agrees to carry the recurring line, access to the data has not been secured, or the business cannot yet write down what a good answer is. The build budget would then be requested on assumptions that nothing allows to be checked.
The first document to draw up is the table of items, filled in for a single use case, with, opposite the operation line, its volume assumptions and the name of the department that will be accountable for it.