It’s not exactly a revelation to say that APIs are a critical element of any modern application design. Developers know that API observability is a great source of metrics and events, even when the API is reporting purely “business” type information (“quantity of shoes sold”, for example). That API data still has value when placed alongside more standard application performance telemetry.
This post will dig into how to take standard JSON output from an API and get it into New Relic using the Flex integration tool. For New Relic users, that’s where the Flex integration tool shows its true power.I’m also taking things a step further by showing you how to integrate with a “real” API—meaning data that’s coming from a web-based system doing a significant amount of processing—using Pi-hole.
Key takeaways:
- APIs serve as vital sources of observability data for monitoring application performance.
- For API observability to be effective, it needs to have proper monitoring tools and agents in place and implemented logging and tracing mechanisms.
- New Relic's Flex integration tool facilitates seamless collection and analysis of API data.
API observability
APIs (Application Programming Interfaces) allow different software systems to communicate with each other, enabling data exchange and functionality sharing between them. They serve as vital sources of observability data for monitoring application performance. By measuring the “four pillars” of API observability—metrics, events, logs, and traces—developers can gain valuable insights into their application's health and troubleshoot any issues that may arise.
API observability vs. API monitoring
The difference between API observability and API monitoring is often a point of confusion. While both are important for maintaining the performance and reliability of APIs, they serve different purposes.
API monitoring focuses on real-time tracking of key metrics to ensure that APIs are functioning properly. This involves setting up alerts for specific thresholds and constantly checking for errors or anomalies. Monitoring allows developers to quickly detect and respond to any issues that may arise with their APIs.
On the other hand, API observability goes beyond just monitoring metrics and also includes deep insights into the behavior and interactions of an API. It provides a holistic view of how an API is performing in production environment by gathering data from different sources such as application logs, network traffic, and user requests. This enables developers to identify potential problems before they arise.
How does the Pi-hole API work?
If you have Pi-hole running, you get to the API by going to http://<your pi-hole url>/admin/api.php?summaryRaw.
The result will look something like this:
{”domains_being_blocked”:115897,”dns_queries_today”:284514,”ads_blocked_today”:17865,”ads_percentage_today”:6.279129,”unique_domains”:14761,”queries_forwarded”:216109,”queries_cached”:50540,”clients_ever_seen”:38,”unique_clients”:22,”dns_queries_all_types”:284514,”reply_NODATA”:20262,”reply_NXDOMAIN”:19114,”reply_CNAME”:16364,”reply_IP”:87029,”privacy_level”:0,”status”:”enabled,””gravity_last_updated”:{”file_exists”:true,”absolute”:1567323672,”relative”:{”days”:”3,””hours”:”09,””minutes”:”53”}}}Let's format the JSON data so it looks a little prettier:
JSON output in "pretty" format
The point is, once we have access to all that JSON-y goodness, it's almost trivial (using Flex integration, which I discussed in this series ) to collect and send into New Relic, to provide further insight into how your network is performing. At that point, you can start to include the information in graphs like this
Graphs of JSON data in New Relic
Assuming you have the New Relic infrastructure agent installed on any system on the network that can access your Pi-hole (and once again, if you need help getting that set up, check out my earlier blog post here), you have relatively few steps to get up and running.
First, the YAML file would look like this (you can also find it on the New Relic Flex GitHub repo in the examples folder.
integrations:
- name: nri-flex
config:
name: pihole_simple
apis:
- name: pihole_simple
url: http://pi.hole/admin/api.php?summaryRaw&auth= #<your API Key Here>
headers:
accept: application/json
remove_keys:
- timestampNext, the New Relic Query Language (NRQL) query you'd need to set up two different charts are as follows:
For the "Query Volume" chart:
From pihole_simpleSample SELECT average(dns_queries_all_replies), average(dns_queries_today), average(queries_forwarded), average(queries_cached), average(dns_queries_all_types) TIMESERIESFor the "Blocking Activity" chart:
From pihole_simpleSample SELECT average(ads_blocked_today), average(domains_being_blocked) TIMESERIESThis is, of course, only the start of the insights you can gain from your Pi-hole server (and by extension, ANY device or service that has an API with endpoints that provide data). If you find additional use cases, feel free to reach out to me in the comments below, on social media, or when you see me at a conference or meet-up.
Why use New Relic for API observability?
There are several reasons why New Relic is the optimal tool for API observability:
- Real-time visibility: New Relic allows for real-time visibility into API performance, enabling teams to quickly identify and resolve issues, optimize performance, and deliver better customer experiences.
- Interactive dashboard: You can create interactive dashboards to visualize and analyze API data from a holistic view. This allows teams to easily monitor key metrics, identify trends, and gain insights into how APIs are being used.
- Proactive monitoring: New Relic's AI-powered alerting system can proactively notify teams of potential issues or anomalies with APIs, allowing them to take action before users are impacted.
- Advanced features: By utilizing New Relic's advanced features such as distributed tracing, error tracking, and dashboards, organizations can gain a deeper understanding of their APIs' health and make informed decisions to improve their overall functionality.
User-friendly: With its beginner-friendly interface and robust features, New Relic is a must-have for any company looking to enhance their API observability and drive business success.
Next steps
A quick look around your application, infrastructure, and even your home office will probably reveal a few things that have an API. That API data has value on its own and when put in context with traditional application performance telemetry. Why not log into New Relic and see what kind of metrics you can collect based on what you learned in this blog?
Don't have New Relic? Become a hands-on observability practitioner by creating a free account. Your free account includes 100 GB/month of free data ingestion, one free full-platform user, and unlimited free basic users.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.
