Optimizing Streaming Media Performance: 7 Things You Need to Monitor

Monitoring the performance of streaming video can be even more complex than traditional application performance management of websites, mobile apps, e-commerce apps, and even single-page applications.

For one thing, the streaming video user experience depends on a complex interaction of the web page, the media content, and the media player, each of which may be “owned” by different parts of the organization. While users care only about how long it takes to begin watching their video, the folks who “own” the webpage may not have any control over the media player, the people behind the player may not care about the webpage, and the person who owns the content may not care about either.

What if a user is looking for a title to view, for example, and the search functionality doesn’t work? Or what if the page is so slow to load that visitors abandon it even before they get to the media? And what if the player’s performance isn’t good enough to deliver a smooth streaming experience free of stutters and glitches?

Different than transactional websites

To truly understand what’s happening, you have to grasp the context of the entire platform. There are a lot of components involved in delivering a great streaming media experience, and they all have to work properly and work well together.

If you think about an e-commerce use case, for example, the event stream is fairly straightforward. A user visits a site or app, it loads (1 event), the user searches for products or clicks on a product (1 event), the product page loads (1 event), the user adds the item to their shopping cart (1 event), the checkout page loads (1 event), the user checks out (1 event), the checkout is processed and a notification is sent (1 event).

For streaming applications, the story is a little different. The page load, player load, media load, and so on can be counted as a single event, but what most companies want to do is to measure the heartbeat of their stream. This could happen every second, every five seconds, or whatever period is relevant to their needs. Additionally, buffering, bitrate changes, and errors can add to the event count. The event count can get quite massive when there are concurrent streams.

How to use New Relic to monitor streaming video performance

New Relic is known to be great at visualizing transactional applications, such as e-commerce purchases. But how do you monitor streaming video and represent streaming Quality of Service (QoS)?

With the New Relic One, it’s not that hard. When you put it all together, it’s all still timeseries data. So it just requires reducing the noise and focusing on the basics. For example, here are some QoS KPIs you should care about that you can monitor using New Relic:

1. Playback errors: Did the player/stream fail to load?
2. Startup time: What is the page load time, player load time, and time to first frame?
3. Buffer events: How many times/how long is the stream stalled while playing?
4. Video quality (bitrate) change: Is the stream degrading in quality?
5. Companion ads: Similar metrics to streaming video, but focused on the third-party ad stream (e.g., ad load, ad start, ad end, ad quality, etc.)
6. CDN data: This is a little more advanced, but with Content Distribution Networks such as Akamai, Cloudflare, and so on, you can pass debug headers to figure out if a piece of content can be cached, the request ID, what edge did it hit and was it efficient, and where it was in the cache lifecycle.
7. Milestone events: Was the content viewed 10%, 25%, 50%, 100%?

To capture these events, you use existing New Relic APIs to instrument the player or connected device. So, whether you’re streaming via HTML5, a PDK player, Roku, Chromecast, or whatever, it’s all the same. Most companies want to query against New Relic metadata to see any degradation in their streams. They want to see what cities had the highest re-buffering counts. They want to know what page URLs had the highest player errors. They need to find out what user IDs had the most bitrate changes, or what CDNs are performing poorly—grouped by URL.

This plain and simple approach effectively reduces the amount of metric noise potentially created by the diversity in players and CDNs. The majority of players and libraries have these events—you merely need to use your APIs to publish events to New Relic Insights to get the information they need.

Don’t forget the ROI

This approach can have big dollar implications for media companies. A media outlet that serves 5 million page views a day but suffers an 8% bad impression rate at a $20 CPM loses $17.5 million a year. If the company could cut that bad impression rate in half (to 4%), it could save more than $8 million a year. And the ROI is even better for video ads, which typically carry significantly higher CPMs.

Clearly, there are big rewards for media companies to instrument their streaming video and ads with New Relic APM. Fortunately, doing so can be surprisingly straightforward. Now that you know it’s possible, give it a try, or reach out to us for more information at newrelic.com/contact-sales.

Additional resources:

• New Relic Can Monitor Video and Audio Performance (blog post)
• Keep Your Customer Experience Streaming (white paper)