In order to create effective machine learning models, it’s necessary to scrape vast amounts of data, but truly understanding your incoming data in terms of annotation and label distribution for every segment can be very challenging. It can also be difficult to maintain multiple dashboards of ML models in production. In order to update your models effectively, you need a system that can monitor and alert you when your models aren’t working as expected.
New Relic is partnering with Superwise to help data scientists, data engineers, and machine learning practitioners automate model monitoring at high scale and gain insights at the most granular levels so they can pinpoint issues and drive actions to maximize business outcomes.
In this tutorial, you’ll learn how to set up and integrate Superwise with New Relic Alerts and Applied Intelligence to correlate model and infrastructure alerts, and dive deeper to investigate model performance.
The following video shows the functionality you’ll have once you integrate Superwise with New Relic One.
Why integrate Superwise with New Relic One?
Superwise is a model observability platform for leveraging machine learning at high scale. You can use the platform's automated KPIs, incidents, and model customization builder to deliver critical insights out-of-the-box. By integrating Superwise with New Relic One, you provide your data teams with full visibility and analysis of your models.
In this dashboard, MLOps engineers can see the general health of their ML models, including which model is active, which one is drifting, and where there are issues.
Integrating Superwise with New Relic One
To set up the integration you need to do the following:
- Log in to one.newrelic.com, go to Explorer and select + Add more data.
- Scroll down to the MLOps Integration section and select Superwise.
- Select your New Relic account ID.
- You need to create two access tokens. First, select Create a Key under Model metric data. Then select Create a Key under Incidents. Make sure the keys are available for future steps.
- Log in to the Superwise portal and go to notification channel settings. Select New Relic and create a channel name.
- Using the two tokens you created in step 4, paste the API key under Incident Intelligence API and the Insight key under Telemetry Key in the Superwise portal.
- Test both tokens for verification and select Save.
- In the New Relic integration dashboard, select See your data. This will redirect you to an automatically generated New Relic dashboard powered by Superwise.
- Superwise’s dashboard contains three charts: the Model Activity chart, the Model Input Drift detection chart, and the Incident Insights chart.
In this dashboard, MLOps engineers can see the general health of their ML models, including which model is active, which one is drifting, and where there are issues.
- Set up New Relic policies and conditions based on your ML Model metrics by creating an alert condition. To do so, select the three dots on any metric chart in your dashboard and choose Create alert condition from the dropdown.
- You can also correlate your incidents to reduce noise and create custom decisions by training your ML model.
New Relic’s partnership with Superwise will provide your data and engineering team full visibility into ML-powered applications and the future of software by allowing MLOps and DevOps teams to seamlessly collaborate.
Next steps
For more information on how to set up integrations with New Relic MLOps or integrate Superwise in your observability infrastructure, visit the New Relic MLOps integrations page.
And if you’re new to New Relic but interested in digging in, experience the simplicity of New Relic One yourself by signing up for a forever free account.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.
