Start with the budget
A 99.9% objective over 30 days sounds strict, but it is simply an allowance. 30 days is 43,200 minutes, and 0.1% of that is 43.2 minutes of complete failure. Measured by requests, a service handling 10 million requests in that window (an example figure) may fail 10,000 of them.
| Objective | Budget share | Over 30 days at full outage |
|---|---|---|
| 99% | 1% | 432 minutes (7.2 hours) |
| 99.5% | 0.5% | 216 minutes (3.6 hours) |
| 99.9% | 0.1% | 43.2 minutes |
| 99.95% | 0.05% | 21.6 minutes |
| 99.99% | 0.01% | 4.32 minutes |
Two things become clear from this table. Each additional nine costs ten times less room to make mistakes, so a target should reflect what users need and not what sounds impressive. Also, at 99.99%, no human can respond to a page and fix a full outage in time, so that target implies automation, redundancy and a very careful rollout process. A target you cannot defend is a number nobody will respect.
The budget also changes the conversation with product and engineering. It is a shared quantity that can be spent on releases, experiments and migrations, and a clear error budget policy states what happens when it runs out.
Why threshold alerts on error rate fall short
Alerting on "error rate above 0.1%" treats a 30-second blip and a 12-hour burn the same way. It pages for the blip that used up a tiny share of the budget, and if you widen the evaluation window to avoid that, it reacts far too slowly when something serious happens.
Burn rate solves this by asking how fast you are using the budget compared with the rate that would use it exactly over the window. A burn rate of 1 uses the whole 30-day budget in 30 days. A burn rate of 10 uses it in 3 days. On a 99.9% objective, burn rate 1 equals an error rate of 0.1%, and burn rate 14.4 equals an error rate of 1.44%.
The four window pairs
Wrentrix SLOs use error budgets and 4 burn-rate window pairs, with multi-window alerts that resolve themselves. The widely cited reference for this design is The Site Reliability Workbook, whose table is shown below for a 30-day window, with the arithmetic worked out. Check the configured values in your own installation, and adjust if your objective calls for it.
| Burn rate | Long window | Short window | Budget used at trigger | Error rate at 99.9% | Time to empty budget | Action |
|---|---|---|---|---|---|---|
| 14.4 | 1 hour | 5 minutes | 2% | 1.44% | 50 hours | Page |
| 6 | 6 hours | 30 minutes | 5% | 0.6% | 5 days | Page |
| 3 | 1 day | 2 hours | 10% | 0.3% | 10 days | Ticket |
| 1 | 3 days | 6 hours | 10% | 0.1% | 30 days | Ticket |
The maths checks out as follows: 30 days is 720 hours. Budget consumed equals burn rate times the long window divided by 720. For the first row, 14.4 × 1 ÷ 720 = 0.02, or 2%. For the second, 6 × 6 ÷ 720 = 0.05. For the third, 3 × 24 ÷ 720 = 0.1. For the last, 1 × 72 ÷ 720 = 0.1. Time to empty is 720 hours divided by the burn rate.
Each row answers a different question. The first asks whether something is burning quickly right now. The last asks whether something is leaking slowly that nobody would otherwise notice. The burn-rate alerts guide explores these trade-offs, including how to scale for shorter windows.
What the short window does
The short window is the part people tend to skip, and it is why these alerts behave well. An alert fires only if both the long and the short window exceed the threshold.
Here is an illustrative timeline for the first row, on a 99.9% objective. A deployment at 10:00 makes 5% of requests fail, and a rollback at 10:20 fixes it.
- At about 10:17, the error ratio over the past hour reaches 1.44%. That is 5% for 17.3 minutes out of 60, since 5 × 17.3 ÷ 60 ≈ 1.44, assuming the earlier part of the hour was clean. The 5-minute window is well above threshold too, at 5%. The page goes out.
- At 10:20, the rollback completes, so failures stop.
- At about 10:25, the 5-minute window drops to zero. The alert resolves, even though the 1-hour window is still elevated (it holds 5% × 20 minutes, a ratio of about 1.67%, until the bad period starts ageing out at 11:00).
Without the short window, the alert would stay red until about 11:03, and whoever just rolled back would have no way to tell from the alert that the fix had worked. With it, the alert closes within minutes. That self-resolution is the behaviour you want: the page tells you when the problem has really stopped, and it removes a whole category of "is this still going on?" noise. It also pairs with automated remediation, since a successful action shows up as an alert that closes.
Choosing what to measure
The alerting mechanics are the easy part. The hard part is the indicator. Some advice that holds in most systems:
Measure what the user experiences as close to the user as you can. A database that is up, with an application returning 500s, is an outage. Request-based indicators on an edge endpoint, or on the service nearest the customer, are generally better than host-level availability.
Define "good" explicitly. A common latency SLI is the proportion of
requests served under a threshold, for example "99% of POST /orders finish
under 800 ms". It behaves like an error rate, so it fits the same budget arithmetic. An SLO
written on a p99 value directly is harder to reason about, because percentiles from
different windows can't be combined.
Decide what is excluded. Are 4xx responses failures? A 429 you return under load usually counts against you. A 404 for a mistyped URL should not. Write this down, since it will be questioned later.
Do not create one SLO per endpoint. Group by what users are trying to do: log in, search, pay. A handful of SLOs that people look at beats hundreds that nobody does. If a service has very little traffic, the percentages jump around, and a single failed request out of 50 is already a 2% error rate. Use longer windows for those services or synthetic traffic from probes.
Watch for sampling. If indicators are computed from sampled data, the sampling must be accounted for. An error-biased sample that keeps all failures and 5% of successes would produce a hugely inflated error rate if counted naively. Confirm how any given metric is derived before you trust it for a contractual purpose.
Connecting SLOs to the rest of the platform
An SLO on its own says that something hurts. Getting from the alert to a cause uses other parts of the product. You begin with the endpoint percentiles and error groups in tracing, and with the deviations described under baselines. The problem engine groups signals by topology into one incident, so that a single fault does not appear as separate SLO breaches on every upstream service. "Explain this screen" is available on an SLO, and answers using the local model if you have AI enabled.
There is an organisational limit too. Tools can calculate budgets and page accurately. They can't decide who is allowed to freeze a release, or whether a product owner will accept a stated target. Teams that adopt burn-rate alerts without that agreement get accurate numbers that change nothing.
When SLOs are not the right starting point
If your service has no real user traffic, or if you lack basic instrumentation, write those first. An SLO for a batch job that runs nightly is better expressed as "completed by 06:00" than as a request ratio. A newly launched system with an unknown baseline also benefits from a few weeks of observation before anyone commits to a number. Finally, if a team already has a set of alerts it trusts, and a low page volume, there is no urgent need to redo them. Introduce the change service by service.
Frequently asked questions
What is an error budget?
It is the amount of failure an SLO allows over its window. A 99.9% objective over 30 days allows 0.1% of requests to fail, which is equal to 43.2 minutes of total outage. Teams can spend that allowance on releases and risk, and a written policy decides what happens when it runs out.
What does a burn rate of 14.4 mean?
It means you are using the budget 14.4 times faster than the rate that would empty it exactly at the end of the window. On a 30-day window, that empties the budget in 50 hours, and one hour at that rate uses 2% of it. On a 99.9% objective, it corresponds to a 1.44% error rate.
Why use two windows for one alert?
The long window shows that a meaningful share of the budget was actually spent, so blips do not page. The short window shows the problem is still happening, which lets the alert resolve soon after the fix. Together they give you both precision and quick recovery.
What is a good SLO target to start with?
Base it on what the service has actually delivered recently and what users need, and not on a round number with more nines. A target slightly tighter than your current measured performance is a practical starting point. Review it after a month or two, since an initial target is rarely correct.
Should every SLO alert page someone?
No. In the common four-pair design, the two fast pairs page, and the two slow pairs open tickets. A slow leak needs attention during working hours, not at 03:00. Paging only on what needs immediate action is one of the main ways to reduce alert fatigue.