Collecting custom JSON metrics using the New Relic Flex integration

In my previous post in this Flex series, Absolutely simple metrics with Flex I introduced Ookla's internet Speedtest command line utility and showed you how to pick up the latency, download bps, and upload bps statistics by using a Python script.

Eagle-eyed readers might have noticed a tantalizing element of the command we tested with:

speedtest --accept-license -f csv

That -f csv (format the output for CSV) implies other formats. And indeed, if you invoke the help option (-h) you see that multiple output formats are supported:

All the cool kids use JSON for speed testing

It just so happens the Flex integration automatically ingests and proceses JSON information. And since so many tools and techniques we IT practitioners use, from external APIs to internal utilities, have adopted JSON as one of their output options, it’s worth re-visiting this example and re-implementing it with a JSON twist.

In the case of our example, the Speedtest JSON output looks like this:

In many ways, this version is much simpler than the script-based version in my Absolutely simple metrics post. I'll call out some of the key elements of this YAML file:

interval: 1h As with the script-based version, we don't want to saturate our bandwidth with tests every few minutes, so setting the interval to once per hour seems reasonable. Feel free to "salt to taste."

- run: speedtest --accept-license -f json

Here's the actual command we're running, with JSON specified (not .jsonl* or .jason-pretty):

timeout: 300000

As with the script-based version of this monitor, running Speedtest takes more than the default 30 seconds, so we need to extend the running time to allow for it. Five minutes (500000 milliseconds) has worked fine in my tests.

Getting set up

To quickly review, here's what you'll need to do before you start this tutorial:

• Have a New Relic account. (The free one will work fine!)
• Install the New Relic infrastructure agent as I described in Absolutely simple Flex.
• Install the Ookla Speedtest utility.

YAML lama ding dong

With those particulars out of the way, now we're ready to set up our YAML file in /etc/newrelic-infra/integrations.d (or C:\Program Files\New Relic\newrelic-infra\integrations.d if you're testing this on a Windows system).

If you look back at the sample JSON output, you'll notice one of the first items is a timestamp. Left in place, this could conflict with the New Relic internal time marker during ingest, so we're better off removing it using the remove_keys function.

For details on supported YAML functions (like this remove_keys option) check out this document on data parsing and transformation functions. For information on other commands and switches, see our documentation on Flex YAML options.

No, I didn't know what "JSONL" format was either. But running it once showed me quickly—it's a list of each test as it executes. Great for tracking progress, but not so good for metric collection.

Now add the NRQL

After your YAML file is set up and running, data should be flowing into your New Relic account. To see it, let's start again with the very high-level query:

FROM speedtestSample select *

This will show all of the JSON information we're collecting:

If you scroll right, you'll find values for Download.Bandwidth, Download.Bytes, Download.Latency, Upload.Bandwidth, and more. This should be everything you need, with one hitch: in the script-based version of this monitor, we were able to divide the download number by 125,000 to get the bits per second (bps) value before presenting that output to New Relic for ingest.

In this case, because we're taking the unprocessed JSON data, we'll have to post-process it. Therefore, our final NRQL query will look like this:

FROM speedtestSample select average(download.bandwidth)/125000, average(upload.bandwidth)/125000 TIMESERIES

That query, when turned into a chart, makes a lovely addition to the system's dashboard: