It’s not exactly a revelation to say that APIs are a critical element of any modern application design. Whether, as a developer, you’re using someone else’s API or allowing interactions with your app via one, APIs are part of the fabric of good coding practices. Users of New Relic (and observability in general) also know that APIs are a great source of metrics and events. Even when the API is reporting purely “business” type information (“quantity of shoes sold”, for example), that API data has value when placed alongside more standard application performance telemetry.  This post will dig into how to take standard JSON output from an API and get it into New Relic using the Flex integration tool.

For New Relic users, that’s where the Flex integration tool shows its true power. And that’s what I’ll be showing you in this blog post. I’ve written about Flex before. And I’ve written about collecting data that comes to you in JSON format using Flex, too. In this article I’m taking things a step further by showing you how to integrate with a “real” API—meaning data that’s coming from a web-based system doing a significant amount of processing.

For my example application, I’m using Pi-hole. Pi-hole acts as a first-level Domain Name System (DNS) server for your network. It filters out malicious, unnecessary, or just plain inappropriate content—not just at the URL level (meaning blocking an entire web page/website), but at the individual component level. So it may block one graphic or embedded video on a page, but allow several others through. 

You may not think many web pages are like this, but the truth is websites use these links all the time. Here's the statistics from my house on a typical day

DNS query statistics as displayed on a pi-hole server

My point is that this system is processing A LOT of data constantly, and performance is a critical aspect to monitor. Which makes it a perfect choice to provide an example for this blog

How does the Pi-hole API work?

If you have Pi-hole running, you get to the API by going to http://<your pi-hole url>/admin/api.php?summaryRaw.

The result will look something like this:

{”domains_being_blocked”:115897,”dns_queries_today”:284514,”ads_blocked_today”:17865,”ads_percentage_today”:6.279129,”unique_domains”:14761,”queries_forwarded”:216109,”queries_cached”:50540,”clients_ever_seen”:38,”unique_clients”:22,”dns_queries_all_types”:284514,”reply_NODATA”:20262,”reply_NXDOMAIN”:19114,”reply_CNAME”:16364,”reply_IP”:87029,”privacy_level”:0,”status”:”enabled,””gravity_last_updated”:{”file_exists”:true,”absolute”:1567323672,”relative”:{”days”:”3,””hours”:”09,””minutes”:”53”}}}

Let's format the JSON data so it looks a little prettier:

JSON output in "pretty" format

The point is, once we have access to all that JSON-y goodness, it's almost trivial (using Flex integration, which I discussed in this series ) to collect and send into New Relic, to provide further insight into how your network is performing. At that point, you can start to include the information in graphs like this

Graphs of JSON data in New Relic

Assuming you have the New Relic infrastructure agent installed on any system on the network that can access your Pi-hole (and once again, if you need help getting that set up, check out my earlier blog post here), you have relatively few steps to get up and running.

First, the YAML file would look like this (you can also find it on the New Relic Flex GitHub repo in the examples folder.

 

integrations:
  - name: nri-flex
    config:
      name: pihole_simple
      apis:
        - name: pihole_simple
          url: http://pi.hole/admin/api.php?summaryRaw&auth= #<your API Key Here>
          headers:
            accept: application/json
      remove_keys:
        - timestamp

Next, the New Relic Query Language (NRQL) query you'd need to set up two different charts are as follows:

For the "Query Volume" chart:

From pihole_simpleSample SELECT average(dns_queries_all_replies), average(dns_queries_today), average(queries_forwarded), average(queries_cached), average(dns_queries_all_types) TIMESERIES

For the "Blocking Activity" chart:

From pihole_simpleSample SELECT average(ads_blocked_today), average(domains_being_blocked) TIMESERIES

This is, of course, only the start of the insights you can gain from your Pi-hole server (and by extension, ANY device or service that has an API with endpoints that provide data). If you find additional use cases, feel free to reach out to me in the comments below, on social media, or when you see me at a conference or meet-up.