Service-level metrics

How often are outages occurring that affect customers and end users? Survey results showed that:

• Outages still happen fairly frequently, but the number of respondents who said they happen once per week or more decreased year-over-year (YoY) by 36% for high-business-impact outages, 52% for medium-business-impact outages, and 63% for low-business-impact outages.
• Low-business-impact outages happen the most frequently (53% noted once per week or more).
• High-business-impact outages happen the least frequently (two to three times per month or fewer), but almost a third (32%) still experience them once per week or more, and 13% still experience them once a day or more.

Given the relative frequency of outages, the findings of how often manual effort and incident tickets are the sources of knowledge for these outages are noteworthy.

Like last year, organizations with full-stack observability (by the report’s definition) consistently have fewer outages than organizations without full-stack observability.

When we looked at the mean time to detect an outage, a common service-level metric used in security and IT incident management, the survey results showed:

• The most commonly cited MTTD across all business impact levels was 5–30 minutes.
• High-business-impact outages tend to take the longest to detect, with 44% saying it takes 30+ minutes and 21% saying it takes at least 60 minutes.
• MTTD improved in general YoY across all impact levels—for example, it was 8% faster YoY for high-business-impact outages.

Respondents with full-stack observability (based on our definition) were once again more likely to experience the fastest MTTD (less than 30 minutes). They also saw the most MTTD improvement. For example, those with full-stack observability were 19% more likely to detect high-business-impact outages in 30 minutes or less, compared to those without full-stack observability.

In addition, those with more capabilities deployed had a faster MTTD. For example, respondents who said their organization has 5+ observability capabilities currently deployed were 40% more likely to detect high-business-impact outages in 30 minutes or less, compared to those with 1–4 deployed.

We see similar patterns with MTTR, another common service-level metric used in security and IT incident management:

• The majority had an MTTR of at least 30 minutes across all business impact levels.
• High-business-impact outages tend to take the longest to resolve, with 60% saying it takes more than 30 minutes and 34% saying it takes more than an hour to resolve.
• MTTR improved in general YoY; for example, it was 26% faster YoY for high-business-impact outages.

Respondents with full-stack observability (based on our definition) were once again more likely to experience the fastest MTTR (less than 30 minutes). They also saw the most MTTR improvement YoY. For example, those with full-stack observability were 18% more likely to resolve high-business-impact outages in 30 minutes or less compared to those without full-stack observability.

And those with five or more capabilities deployed had a faster MTTR. For example, respondents who said their organization has 5+ observability capabilities currently deployed were 42% more likely to resolve high-business-impact outages in 30 minutes or less compared to those with 1–4 deployed.

This year, we wanted to see how much revenue critical business application outages cost organizations per hour of downtime, on average. We also estimated the annual outage cost based on high-business-impact outage frequency, total downtime (MTTD and MTTR), and hourly outage cost.

Hourly outage cost

As far as how much survey respondents said critical business app outages cost on average per hour of downtime, we found that:

• Three in five (61%) said they cost at least $100,000, 32% said at least $500,000, and 21% said at least $1 million per hour of downtime. 
• A quarter (25%) said they cost less than $100,000 per hour of downtime.
• Notably, 12% said they weren’t sure how much these outages cost.

In addition, the cost of outages is higher for those who don't have full-stack observability or a mature observability practice. For example, 42% of those from organizations with full-stack observability or a mature observability practice (by the report’s definitions) said critical business app outages cost less than $250,000 per hour of downtime compared to 35% of those without full-stack observability, and 37% of those without a mature observability practice.

We wanted to know how respondents thought their organization’s MTTR for outages had changed since adopting an observability solution. We found that:

• About two-thirds (65%) said their MTTR had improved to some extent, including 31% who said it improved by 25% or more.
• Only 16% said their MTTR had worsened to some extent.
• Just 14% said it remained the same.

Several factors were correlated to a greater likelihood of the best MTTR improvements, including:

• Deploying five or more observability capabilities: 68% compared to 40% for those with zero and 45% for those with 1–4 capabilities deployed.
• Employing five or more observability practice characteristics: 69% compared to 61% for those with 1–4 characteristics employed.
• Having a mature observability practice (by the report’s definition): 68% compared to 64% without (56% more likely to experience MTTR improvements of 25%+).
• Achieving full-stack observability (by the report’s definition): 68% compared to 63% without (27% more likely to experience MTTR improvements of 25%+).
• Spending at least $100,000 per year on observability: 67% compared to 62% who spend less than $100,000 per year and 17% who spend nothing (the more they spend, the more likely they were to see an improvement).

Those who said they get the most value from observability ($2.5 million or more) were more likely to say their MTTR has improved since adopting it.