Definition of the error budget
A service level objective set below 100% allows a share of failure. The error budget measures this share. The “Site Reliability Workbook”, which Google publishes online (sre.google/workbook), calculates it by subtraction: 100% minus the SLO.
The “Embracing Risk” chapter of the book “Site Reliability Engineering” (sre.google/sre-book) presents it as a “clear, objective metric” of the unreliability allowed to a service over a quarter. The SLO on which this calculation depends is defined in our entry SLO, SLI and SLA.
This chapter starts from a tension. The development team is judged on the pace of its deliveries, the team responsible for reliability on the stability of the service: our entry on DevOps describes this siloing. The error budget gives them a criterion accepted in advance. In the loop described by the chapter, production releases continue as long as some budget remains; once it is spent, they stop while the system is made more robust.
Calculating the error budget: an example in requests and in minutes
Take a carrier’s parcel tracking service, with an SLO of 99.8% successful requests over a rolling 28-day window. These values are calculation assumptions, not recommended values.
- As a proportion: 100% minus 99.8%, giving a budget of 0.2%.
- In events: for 5,000,000 requests received over the window, 0.2% represents 10,000 failed requests.
- In time, at constant traffic: 28 days contain 40,320 minutes, 0.2% of which is 80.64 minutes, or about 1 h 21 min of complete outage.
An incident that causes 4,000 requests to fail therefore consumes 40% of the budget. The Workbook calls the pace of consumption the burn rate: at 1, the budget is exhausted exactly at the end of the window. If the failure rate held at 2%, ten times the allowed rate, the burn rate would be 10 and the budget would be emptied in 2.8 days.
The error budget policy
The budget settles nothing without a written rule (error budget policy). According to the Workbook, it specifies the actions to take when the budget is consumed and who takes them; the product owner, the development team and the SRE team approve it. Without such a commitment, the book adds, meeting the SLO remains one tracking indicator among others.
The example policy published as an appendix to the Workbook provides for:
- freezing changes and production releases until the service is back within its objective, except for the highest priority defects and security fixes;
- a post-incident review as soon as a single incident consumes more than 20% of the budget over four weeks;
- escalation up the hierarchy in the event of disagreement on the calculation or on the actions.
The “Embracing Risk” chapter also mentions graduated responses, such as slowing down deliveries as the budget nears exhaustion.
Error budget, SLO, RTO, user satisfaction: not to be confused
The error budget and the SLO are derived from each other: setting the objective sets the budget.
The budget does not measure user satisfaction: the Workbook sees it only as an approximation. Over thirty days, a four-hour outage, four one-hour outages and a constant error rate of 0.5% probably do not cause the same amount of dissatisfaction; the budget nevertheless counts them in the same way.
The maximum time to recover after a disaster is set elsewhere, by the RTO in our entry RTO RPO.
Nor is the budget a sanction: the Workbook’s example policy states that it is not intended to punish a missed objective. The other practices of the discipline appear in our entry SRE.
For writing this policy with business and technical teams, then aligning alerts with budget consumption, see our page SRE and observability.