Muting Rules Now Available in New Relic Alerts

During times of planned system disruptions, a steady stream of noisy, unnecessary alerts can be a major distraction. You need to find a balance that allows you to filter out the noise yet still maintain observability and alerting on the rest of your system.

With muting rules in New Relic Alerts, you can silence notifications during planned disruptions like maintenance windows, deployments, and testing.

By applying muting rules, you’ll minimize the risks of overmuting by creating rules that identify violations using all of the contextual metadata we know about the entity and signal, which provides more depth than merely enabling or disabling alert conditions. This way, you can target your system’s specific subsets that are being affected by planned disruptions.

Additionally, by keeping alert conditions running but silent during these periods, you can continue to observe when entities have recovered from maintenance and have returned to a healthy state.

You create and manage muting rules using either New Relic Alerts UI or the NerdGraph API explorer.

What is a muting rule?

A muting rule overrides alert policies and conditions you’ve defined in New Relic, so you can suppress notifications during times of known system disruptions. A muting rule contains a set of conditions that match a large number of attributes and tags present in a violation event, including:

• Alert policy and condition names
• Application and service names
• Host names
• AWS tags
• Kubernetes tags
• Any other tags associated with your entities or open telemetry

How muting rules work

Muting rules tell New Relic how to identify individual violations after they are created, but before an incident is opened. They override the default life cycle of an alert to indicate that it should not send notifications.

With muting rules, your normal alert incident life cycle is maintained; the only change is whether or not an incident triggers a notification. Violations and incidents are still opened, and health indicators show the current state of your entities as you progress through your maintenance window.

When the conditions defined in a muting rule match a violation event, the violation is marked as “muted.” When a violation is muted, the incident life cycle is modified in the following ways:

• When a muted violation opens an incident, an “open incident” notification will not be sent.
• If any other violation that is not muted is associated with that incident, then an “open incident” notification will be sent.
• If an incident sends an “open incident” notification, then all subsequent events will trigger notifications.
• If an incident never sent an “open incident” notification, then subsequent events will not trigger notifications.

(Tip: You can find a detailed workflow chart in the docs.)

Enterprise-scale issue detection and incident management require flexibility and automation. Muting notifications is the first in a series of life cycle override actions that we will be releasing this year, including allowing you to schedule muting. All of these controls will be manageable through our API and UI.

How to set up a muting rule

Muting rules are available to everyone using New Relic.

To set up a muting rule:

1. Navigate to  New Relic One > Applied Intelligence.

   

2. From the left-hand navigation, select Muting rules.
   

   
   

   Note: Since New Relic One is a cross-account experience, the Muting rules page lists all muting rules that exist across all of the accounts to which you have access. However, a muting rule currently only operates in one account at a time.

3. Set up your rule:
   1. Click Add a Rule.
   2. Choose a name and description, and select the appropriate account.
   3. Build a violation filter to set conditions that describe the criteria used to match the attributes of a violation object.
   4. Enable the rule when you are ready for it to be active.
      

      
      

    

You can view violations and incidents that have been muted in New Relic Alerts.

Creating complex rule filters

Use complex rule filters for greater precision when muting alerts. For example, if your maintenance only touches specific hosts within a cluster, a subset of services on a host, or if you’re making updates to a data center or availability zone that affects a large number of separately monitored services, complex rule filters can help ensure you only mute the relevant alerts.

Create complex rule filters by combining sets of matching conditions. Filters can either use operators like or or and to combine conditions, and most attribute evaluations can accept an array of possible values.

Creating and managing muting rules with NerdGraph

NerdGraph is an interactive, browser-based IDE for running GraphQL requests. With NerdGraph, you can create and manage muting rules, and perform the same advanced functions available in the UI, including creating muting rules for specific hosts or products and fetching rules for a specific account. Find multiple NerdGraph API examples in the documentation.

Try it now!

The ability to mute notifications is one way we give you more control over your monitoring and help reduce alert fatigue.

If you’re ready to give it a try, sign up for a trial and get started for free.