Guess Less: New Relic Dynamic Baseline Alerts Go GA

Here at New Relic we believe that if you’re not using alerts, you are not really monitoring. In fact, alerts are one of the most popular ways our customers interact with New Relic (after the New Relic user interface and our APIs).

Our customers rely heavily on alerts to keep them notified when anything happens in their digital environment. But creating effective alert conditions is often considered as much art as science, since not only do you need to fully understand the application’s behavior in order to create the best alert thresholds, you also need to know about any seasonal changes or expected trends in the application’s behavior (e.g., regular backups, expected spikes, etc.) so you are not disturbed by alerts on normal behavior.

Static thresholds are great when you know the nature of an application and its normal performance behaviors aren’t likely to change anytime soon. But operations teams often manage many applications with different seasonal patterns and growth trends, and it can be difficult to set thresholds for normal behavior of those applications. In cases like this, dynamic thresholds based on predicted application behavior, using historical data, are extremely helpful.

Last year at our annual FutureStack16 conference in San Francisco, we announced the limited release of Dynamic Baseline Alerts. In the months since that announcement, hundreds of New Relic customers have tried Dynamic Baseline Alerts and been absolutely amazed by this new functionality.

Today, I am very happy to announce that our Dynamic Baseline Alerts capability is now generally available.

What are Dynamic Baseline Alerts?

Dynamic Baseline Alerts make it easy for New Relic customers to set dynamic alert thresholds on their key application performance metrics, using baselines modeled from historical application metric data.

Once you select the Dynamic Baseline Alerts capability and choose an application, you will see a preview of the metric with the predicted baseline. From the drop-down menu, you can select a metric for that application and see the corresponding baseline. Using the threshold sliders, you can then set how closely you want your threshold to follow the baseline prediction. Selecting thresholds is no longer guesswork, as real-time feedback shows you where and how many violations would have been triggered in the last week if you had previously applied the same threshold settings.

We have been asked many times about how we are able to predict the baselines in the first place. It’s definitely not an easy task to predict “normal” behavior of an application given the variety of application types and the corresponding spectrum of metrics. You can get more details on how we derive our baselines and which algorithms we use in Nadya Duke Boone’s fascinating post on how Dynamic Baseline Alerts Automatically Find the Best Algorithm for You.

What’s so awesome about Dynamic Baseline Alerts?

To create effective alert conditions on an application, you need to understand the application and know what the best top and bottom thresholds are for that application. And, even then, static thresholds do not take into account seasonality changes, changes based on time of day, etc. These issues (static thresholds and setting top and bottom thresholds) are top contributors to alert fatigue.

Using Dynamic Baseline Alerts, New Relic customers can easily

• Create alerts with confidence. Dynamic Baselines are designed to forecast the behavior of the application based on historical data. We have dramatically reduced the guesswork required to come up with best static thresholds—both higher and lower—for a given application. Not only that, once you set the dynamic thresholds, you can instantly see how many alerts you would have received in the past week with the selected values. This helps you quickly understand if your thresholds are too conservative or too relaxed. In short, New Relic’s Dynamic Baseline Alerts let you create alerts with confidence.
• Establish dynamic thresholds—set ’em and forget ’em. Static thresholds don’t work well with seasonal and long-term changes in application behavior (throughput, response time, etc). With Dynamic Baseline Alerts, you can set up thresholds once and be confident that you will be alerted if your application’s performance does not fall within a certain threshold from the predicted baseline. Additionally, as more historical data is collected over time, the accuracy of the projected baselines continues to improve.
• Use minute-scale granularity to see short-lived anomalies. We calculate our baselines with minute-scale granularity. So Dynamic Baseline Alerts algorithms can learn the short-term patterns and seasonalities hiding in your metric data. For instance, if you have an application that runs a backup during the first five minutes of every hour resulting in a slight impact on performance, Dynamic Baseline Alerts will learn to account for that pattern.

Want to learn more?

As you might have guessed, we are very excited about Dynamic Baseline Alerts and have created several pieces of content to help you get the most out of it. My colleagues Nadya Duke Boone and Nate Heinrich have written great blog posts explaining the ins and outs of Dynamic Baseline Alerts, and we’ve also built helpful documentation:

• How We Find the Best Algorithms for Dynamic Baseline Alerts
• The Cool Math Behind the Slider in Our Dynamic Baseline Alerts
• How New Relic Keeps Incidents From Biasing Your Alerting Baselines
• Dynamic Baseline Alerts documentation

In today’s world of artificial intelligence and machine learning, busy APM users should not have to guess at establishing suitable alerting thresholds for their applications nor deal with static thresholds for applications with seasonal behavior changes.

Because alerting is such a core piece of monitoring, it needs to be made easy and efficient. And that’s exactly what New Relic Dynamic Baseline Alerts are designed to do. They let you see a predicted baseline derived from historical application data and visually set dynamic thresholds based on the predicted baseline.

Dynamic Baseline Alerts will begin rolling out today to all customers with a New Relic PRO (or higher) subscription. Go try this powerful capability yourself!