It’s not exactly a revelation to say that APIs are a critical element of any modern application design. Whether, as a developer, you’re using someone else’s API or allowing interactions with your app via one, APIs are part of the fabric of good coding practices. Users of New Relic (and observability in general) also know that APIs are a great source of metrics and events. Even when the API is reporting purely “business” type information (“quantity of shoes sold”, for example), that API data has value when placed alongside more standard application performance telemetry. This post will dig into how to take standard JSON output from an API and get it into New Relic using the Flex integration tool.
For New Relic users, that’s where the Flex integration tool shows its true power. And that’s what I’ll be showing you in this blog post. I’ve written about Flex before. And I’ve written about collecting data that comes to you in JSON format using Flex, too. In this article I’m taking things a step further by showing you how to integrate with a “real” API—meaning data that’s coming from a web-based system doing a significant amount of processing.
For my example application, I’m using Pi-hole. Pi-hole acts as a first-level Domain Name System (DNS) server for your network. It filters out malicious, unnecessary, or just plain inappropriate content—not just at the URL level (meaning blocking an entire web page/website), but at the individual component level. So it may block one graphic or embedded video on a page, but allow several others through.
You may not think many web pages are like this, but the truth is websites use these links all the time. Here's the statistics from my house on a typical day
DNS query statistics as displayed on a pi-hole server
My point is that this system is processing A LOT of data constantly, and performance is a critical aspect to monitor. Which makes it a perfect choice to provide an example for this blog
How does the Pi-hole API work?
If you have Pi-hole running, you get to the API by going to http://<your pi-hole url>/admin/api.php?summaryRaw.
The result will look something like this:
{”domains_being_blocked”:115897,”dns_queries_today”:284514,”ads_blocked_today”:17865,”ads_percentage_today”:6.279129,”unique_domains”:14761,”queries_forwarded”:216109,”queries_cached”:50540,”clients_ever_seen”:38,”unique_clients”:22,”dns_queries_all_types”:284514,”reply_NODATA”:20262,”reply_NXDOMAIN”:19114,”reply_CNAME”:16364,”reply_IP”:87029,”privacy_level”:0,”status”:”enabled,””gravity_last_updated”:{”file_exists”:true,”absolute”:1567323672,”relative”:{”days”:”3,””hours”:”09,””minutes”:”53”}}}Let's format the JSON data so it looks a little prettier:
JSON output in "pretty" format
The point is, once we have access to all that JSON-y goodness, it's almost trivial (using Flex integration, which I discussed in this series ) to collect and send into New Relic, to provide further insight into how your network is performing. At that point, you can start to include the information in graphs like this
Graphs of JSON data in New Relic
Assuming you have the New Relic infrastructure agent installed on any system on the network that can access your Pi-hole (and once again, if you need help getting that set up, check out my earlier blog post here), you have relatively few steps to get up and running.
First, the YAML file would look like this (you can also find it on the New Relic Flex GitHub repo in the examples folder.
integrations:
- name: nri-flex
config:
name: pihole_simple
apis:
- name: pihole_simple
url: http://pi.hole/admin/api.php?summaryRaw&auth= #<your API Key Here>
headers:
accept: application/json
remove_keys:
- timestampNext, the New Relic Query Language (NRQL) query you'd need to set up two different charts are as follows:
For the "Query Volume" chart:
From pihole_simpleSample SELECT average(dns_queries_all_replies), average(dns_queries_today), average(queries_forwarded), average(queries_cached), average(dns_queries_all_types) TIMESERIESFor the "Blocking Activity" chart:
From pihole_simpleSample SELECT average(ads_blocked_today), average(domains_being_blocked) TIMESERIESThis is, of course, only the start of the insights you can gain from your Pi-hole server (and by extension, ANY device or service that has an API with endpoints that provide data). If you find additional use cases, feel free to reach out to me in the comments below, on social media, or when you see me at a conference or meet-up.
Next steps
A quick look around your application, infrastructure, and even your home office will probably reveal a few things that have an API. That API data has value on its own and when put in context with traditional application performance telemetry. Why not log into New Relic and see what kind of metrics you can collect based on what you learned in this blog?
Don't have New Relic? Become a hands-on observability practitioner by creating a free account. Your free account includes 100 GB/month of free data ingestion, one free full-platform user, and unlimited free basic users.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.
