Absolutely simple Flex

When you're just getting started with New Relic, you get a solid foundation for observability right out of the gate. Between the pre-built agents and quickstarts; online resources like documentation, labs, and master classes; training from New Relic University; and the New Relic platform itself, you’ll get comprehensive monitoring and observability in your environment with minimal effort.

But no solution has absolutely everything. As the Princess Bride quote goes, "Anyone who says otherwise is selling something." So what do you do when you have almost everything you need but are short a few critical metrics, events, logs, or traces?

Welcome to part two of our series on Flex.

Why you (probably) need custom monitoring values

For many folks working in IT, especially those who have worked with monitoring solutions before, finding the perfect monitoring solution would mean they can:

• Place a feature request with (one of their) monitoring solution vendors, asking to add the metric. (And then wait. And pray. And maybe scream if they think it will help. Spoiler: it won't.)
• Ask the community if/how they've solved the issue—presuming there's an active community for the monitoring solutions in question.
• Start shopping for yet-another-monitoring-tool to get this value, and maybe see if it will cover some of the other outstanding items on the wish list.

But honestly, the most logical course of action is to first see if the monitoring solution will accommodate custom values, whether they are metrics, events, logs, or traces (MELT). And the reality is that any modern, robust, and effective monitoring tool will do just that. The only question is, "How?"

For New Relic, the answer to that question is Flex. Flex is an application-agnostic integration you can use to instrument (meaning "collect and send into New Relic") any metric you can find.

Persuasion with pictures 

Before I get into how Flex is implemented, I’ll offer a brief view of what you gain for your efforts. Here is a screenshot of what a Flex integration looks like "in the wild."

In this case, I’m monitoring how my system is performing using the internet speed data I got from Ookla's Speedtest.net utility running on my PC, which I added to a dashboard with other relevant data.

In this post, my goal is to explain—in the simplest possible way—how Flex works. In subsequent posts, I'm going to go over how to use it to collect MELT data.

Weird flex, brah 

OK, it's not really that weird. Flex requires you to install the infrastructure agent on a system. I want to be clear though: it's the system you are collecting.

But the infrastructure agent on its own isn't Flex. Flex simply rides along with it.

The magic happens in the integrations.d directory. Any correctly configured YAML file you put there will cause data to go to New Relic. It's just that simple.

At least, it's simple if you think configuring YAML is simple. While not "do subnetting in your head" difficult, it doesn't come as second nature to a lot of folks.

YAML for the easily distracted 

To be clear, I'm not going to explain everything about YAML. I'm only going to tell you what you need to know about YAML in order to make Flex work.

YAML stands for "yet another markup language," which, like most acronyms, tells you exactly NOTHING about it. YAML is similar in a lot of ways to XML or JSON, an insight that provides little comfort to many of us who have an emotionally complicated relationship with those other two systems.

My personal trauma aside, YAML is great for configuration files because it's highly structured. But for that same reason, it can be easy to bork something up because of a small (and hard-to-find) oversight. Here are the ones that might trip you up:

• Pretty much everything you do in a YAML file will be in the form of a key: value pair. Some examples:
  • name: nri-flex
  • timeout: 5m
  • shell: powershell
  • run: (Insert your own command to run)
• Words in keys can be separated by underscores, dashes, or spaces.
• Indents matter, in that you must have them and they must be a certain number of spaces.
• Your indents must be spaces. They cannot be tabs.

Now, those are some things to keep in mind with YAML in general. But specifically for a Flex integration, here are some additional pointers.

• For Flex to detect your YAML file, it has to be in the integrations.d directory.
• The filename has to end with .yml (not .yaml).
• The file needs to have "read" permissions for everyone on the system.
  • On Linux, that means a chmod value of u=rw,og=r or 644.
• The file has to start out with the following lines:

With those guidelines in mind, the best way to understand Flex is to create a Flex integration. 

Before reading on, make sure you have a New Relic account and a system with the infrastructure agent installed.

An absolutely simple Flex sample

I'm going to offer you two sample integrations: one for Linux and one for Windows.

The Linux example, pulled from this online tutorial, will grab some disk statistics using the df command. The Windows example, which comes from our extensive library of examples, will use the output of netstat.

One YAML file to rule them all

1. Navigate to the Flex folder
   1. On Linux, that's: /etc/newrelic-infra/integrations.d.
   2. On Windows, it would be: C:\Program Files\New Relic\newrelic-infra\integrations.d.
2. In the text editor of your choice, create flextest.yml.
3. Copy and paste one of the following configurations into the file based on your operating system.

Linux

Windows

4. Save the file.
5. Wait 10 minutes for the data collection to catch up to your system.

Now head over to one.newrelic.com and you'll see... well, not much. Yet.

Dude where's my data? 

The fact that you're collecting data doesn't automatically mean it's displaying somewhere. That's one of the beautiful things about the New Relic platform: it has the scalability to take all the data you can dream of sending, but the discretion not to stick it all in your face. But the flip side to discretion is that you have an extra step to see the data. This is one of the reasons why we provide quickstarts, which are not only a means of monitoring a specific application or tech stack, but also a collection of dashboards, alerts, and more that expose the most commonly wanted data from that app or stack.

So where do you go to find data when it's not already showing? New Relic Query Language, or NRQL (and yes, that's pronounced "nerkul"), lets you explore all of your data, not just the stuff that's automatically displayed on various dashboards.

For now,  I'll give you enough NRQL to get started. (If you’re interested in learning more, read our documentation.)

Starting from your one.newrelic.com page, select the NRQL icon in the upper right corner.

You should see the NRQL query builder screen:

This is a simple interface to enter queries and see the results. To test it out, type this into the query builder and press enter.  Or select the Run button.

FROM Metric SELECT *

That will give you the first few lines of metric data.

But let's get to the good stuff: our custom flex metrics. To find this, I want to point out that in the .yml file, we named our metric TestFlex.

With Flex, the name of the data set will be that name, along with the word "Sample” tacked to the end of it. So our data set is "TestFlexSample." So let's give it a try:

FROM TestFlexSample SELECT *

Now obviously, you don't want all of those items, just the ones that point to the data—whether that's the available bytes, used bytes, capacity percent, or whatnot. In this case, I want a chart of my available disk space:

FROM TestFlexSample SELECT availableBytes

Now that I have a good set of data, I can turn it into a chart, and place the chart on a dashboard. You'll quickly notice you can't select anything except a JSON or Table output:

This blog isn't about NRQL tips or charting traps, but I'm going to give you a tip up-front: 

• You need to grab average, rather than individual, metric values.
• You need to add the keyword TIMESERIES at the end, which creates a line chart with the comparison mapped over time.

So a chartable version of my NRQL query would be:

FROM TestFlexSample SELECT average(availableBytes) TIMESERIES

Now that I have a working, somewhat boring, graph, I can select Add to Dashboard to add it to a display.

Of course, there are way more possibilities to explore concerning Flex, and that's what I'll be doing in upcoming posts.