Skip to content
Technology transformation expertise firm · ARDIANE Group Innovate, secure, succeed

Industries

Measuring a demand forecast: error, bias, service level, stockouts and overstock

In brief

Demand forecasting estimates the quantities customers will want to buy, by item, by location and by period, in order to decide on purchases, replenishment and stock. It is judged on several measures read together (error, bias, service level, stockouts, overstock): Management Square gives their definition, how to read them and their limits here.

Page contents

Imagine a clothing retailer. Every month, the head of procurement presents a falling forecast error. In the same month, the stores report common sizes missing from the shelves, and management control notes an end-of-season stock heavier than the year before. All three findings are accurate: they do not measure the same thing.

Demand forecasting estimates the quantities customers will want to buy, for a given item, location and period, in order to decide on purchases, replenishment and stock. No single figure says whether it is good. It is judged on five measures read together: forecast error, bias, service level, stockout rate and overstock. Each has a precise definition, a way of reading it and a blind spot.

A demand forecast is defined by its granularity and its horizon

Demand is distinct from sales. An item that is out of stock does not sell, and the till records zero where customers left without buying. The sales history is therefore demand censored by stockouts: a model that feeds on it without correction learns the shortage and reproduces it.

Two parameters then set the difficulty. The first is granularity: sales of a product family across a country and a month are far easier to forecast than those of one reference, in one size, for one store and one week. The granularity that matters is that of the decision, whether the order to the supplier, allocation to stores or replenishment.

The second is the horizon, dictated by the supply lead time. A forecast refined the day before the sale does not help an order placed months earlier: error is measured on the forecast that existed when the decision was taken.

Time series and machine learning: the principle of each family

Time series methods, such as exponential smoothing or ARIMA models, extend the past of each series: its level, its trend, its seasonality. They need little additional data, are easy to explain and remain a solid benchmark on regular sales.

Machine learning methods, boosted decision trees or neural networks, learn across all the series a relationship between sales and explanatory variables: price, promotions, calendar, weather, product attributes. They take advantage of similarities between items, but require variables known in advance for the period to be forecast and ongoing operation (retraining, drift monitoring), following the chain described in our page MLOps and LLMOps.

Neither family wins as a matter of principle: the choice is made on the retailer’s data, with the measures that follow.

Forecast error and bias: definitions and reading

Indicator Definition How to read it What the figure does not say
Mean absolute error (MAE) Average of the deviations between forecast and actual, taken in absolute value, in units sold The usual deviation on a series, in the business unit Nothing comparable between an item sold in thousands and another in tens
Mean absolute percentage error (MAPE) Average, over the lines measured, of the absolute deviation divided by the actual A relative error, comparable from one series to another It is undefined when the actual is zero and runs away on small volumes
Weighted absolute percentage error (WAPE) Sum of absolute deviations divided by the sum of actuals Error relative to total volume: each item weighs according to its sales The state of small references, drowned in the total
Bias Sum of signed deviations (forecast minus actual) divided by the sum of actuals Positive, the forecast exceeds actual sales; negative, it falls short Dispersion: large errors in opposite directions give zero bias

MAPE has another flaw. When the forecast is too low, the error on a line cannot exceed the whole of the actual; when it is too high, it has no ceiling. A method tuned to minimise MAPE therefore tends to forecast low. Weighted error copes with zero sales and mitigates this tendency without eliminating it: minimising an absolute deviation amounts to aiming at the median of sales, often lower than their mean for slow-moving items. Bias is therefore tracked alongside.

The definition of bias is a matter of convention, which is written down once and for all: some teams calculate actual minus forecast, others express it in units without relating it to actuals. A persistently positive bias feeds overstock, a negative bias stockouts. It is read by product family, by point of sale and by process step: measured before and after manual adjustments, it shows whether they improve the statistical forecast or degrade it.

None of these measures has a universal target value: the achievable error depends on granularity, horizon and the regularity of sales. The useful comparison sets the method against a naive forecast, for example sales in the same week a year earlier.

Service level, stockouts, overstock: measuring the consequences

The next three measures describe what the retailer did with the forecast.

Service level covers two common definitions, not to be confused. Cycle service level is the share of replenishment cycles completed without a stockout. Fill rate is the share of the quantities demanded that were served from available stock. For the same stock, the two figures differ: a cycle counts as failed as soon as the first unit is missing, whereas the second rate counts only the missing units. The objective is set by category: its cost in safety stock grows faster and faster as full availability is approached.

Stockout rate here means the share of item and point-of-sale pairs with no saleable stock, at a given moment or over a period; the convention adopted (moment or period, scope counted) is written down, as for bias. It does not give lost sales: that quantity cannot be observed, it is estimated. It also depends on the accuracy of system stock: a positive theoretical stock in front of an empty shelf masks the stockout.

Overstock is measured by cover: stock divided by forecast demand per day (or, depending on the convention, by average past sales), expressed in days of sales. Overstock is the share of stock whose cover exceeds a threshold set by category, according to the product’s commercial lifespan. The figure does not identify the cause: forecast too high, supplier minimum order, pack size, or a purchase decided against the forecast.

Forecasting too low and forecasting too high do not have the same cost: on one side a missed sale, on the other unsold stock, markdown and tied-up cash. When a collection is not replenished during the season, the trade-off is made at purchase. Our page retail and luxury describes the stock and product master data foundations on which it depends. This asymmetry is what makes a forecast expressed as a range, with several probability levels, worthwhile.

The blind spots of a good error figure

An error calculated on the total of a family lets the deviations between sizes, colours and stores offset one another. The national figure is excellent and the shelf is empty, as in the situation described at the outset.

The forecast also acts on what it measures. An item given a small allocation sells little, and the actual then confirms the low forecast. As long as demand is not estimated during stockouts, the error can stay low while service deteriorates.

The history, for its part, is only worth what it records. If promotions, price changes and store openings are not recorded with their dates, their effects are attributed to seasonality. The reliability of the history of sales, stock and the commercial calendar falls under data quality.

Then there are the definitions. When procurement, sales and finance each calculate their own stockout rate, the meeting is spent deciding between the figures. Setting a single definition per indicator is the purpose of our business intelligence offering.

Before changing method or tool

The first piece of work is done on what already exists, starting from the last closed season. Recalculate the weighted error and bias of the forecast in place, at the granularity and horizon at which orders were placed, then those of a naive forecast on the same scope. Set the stockout rate and stock cover of the same families alongside them.

This review shows where the forecast is at fault, where the decision departed from it, and on which families trialling another method is worth conducting.

FAQ

Frequently asked questions

Another question? Ask us

What is the difference between demand forecasting and sales forecasting?

Strictly speaking, sales forecasting estimates what will be sold, given the stock available. Demand forecasting estimates what customers would want to buy if the item were present. The two coincide as long as there is no stockout. As soon as an item is missing, recorded sales underestimate demand, and a history used without correction perpetuates the shortage.

MAPE or weighted error: which error indicator should you choose?

MAPE averages the relative errors of each line: it is undefined when actual sales are zero and gives disproportionate weight to items that sell little. Weighted error (WAPE) divides the sum of absolute deviations by the sum of actual sales, which copes with zero sales and weights each item by its volume. On an assortment where many references sell little, the second is safer, provided bias is tracked alongside it.

Does machine learning forecast better than time series methods?

Not as a matter of principle. Machine learning exploits explanatory variables (price, promotions, calendar) and similarities between items, which helps it on recent or heavily promoted references. Time series methods remain a solid benchmark on regular sales. The comparison is made on the retailer's own data, at the granularity and horizon of the decision, against a naive forecast.

How do you forecast demand for a new product with no history?

By analogy. The forecast draws on the past sales of items comparable in category, price, season and in-store exposure, then is corrected from the first weeks of sales. A machine learning model can formalise this analogy from the product's attributes. Uncertainty remains greater than for an established item, which justifies cautious initial quantities when replenishment is possible.

A project to bring to completion?

A first conversation is enough to understand your challenge, identify the right people and tell you frankly what we can bring.