Import all your custom data with just one tool

If you’re in Ops, you might be familiar with searching for the one, true monitoring solution that doesn’t require you to purchase multiple tools to do everything you need. To the point where you might be ready to start chanting: 

Luckily, you don’t need to become a sylvan elf soothsayer to bring all your custom data into New Relic. We offer over 500 quickstart bundles in New Relic Instant Observability that integrate with tools like Cloudflare, Jenkins, and Postman.

If you’ve worked in IT for more than a few weeks, you’ll realize that no vendor can ever predict—let alone collect—all the relevant and necessary data your organization needs to have full visibility into your operations. 

So how do you monitor data from your organization-specific, home-grown, artisanal applications? New Relic gets you the data you need based on your platform, framework, and stack. But your specific metrics, events, logs, and traces may be far too custom for our quickstarts to address everyone’s exact use case. 

There’s also the consideration of shrink-wrapped tools. Even within well-known apps like Exchange, ServiceNow, or SAP there’s a wide range of data that different teams and organizations use. But it goes further than that: I’m talking about environmental solutions for temperature and HVAC, local backup tools, cable and network management systems, and more. 

All of these applications provide valuable insight into the performance of the organization or even act as a bellwether for when things may be headed toward a problem. So, if you can’t find the right solution for your internal or external needs, you might need something customized to your exact needs. New Relic has the ability to do all of that (and more) with a lightweight, application-agnostic solution that can pull in custom data. Best of all:  it comes bundled with our infrastructure agent. We call this solution Flex.    

What is Flex? 

Flex is an all-in-one tool that can take an input, process it, and send it to New Relic as if it came from one of our pre-built integrations.

How does Flex work? 

Flex uses our infrastructure agent and a config file to generate the data you want to report in a format that New Relic can use. You can collect the output from CLI commands; read and process new entries in files at regular intervals; pull data from external URLs or API calls; and take any or all of that and send it into your New Relic account to display on dashboards, as part of an alert trigger.

When should I use Flex? 

Want to capture the longest wait time to close tickets in your in-house ticketing system? Flex. Want to see the top incoming email addresses not in the address book to filter out phishing attempts? Flex. Need to correlate web data with the output from a SQL query and also data hanging out in a .csv? Flex. 

Flex can also monitor services running only in Kubernetes and give you insight into the performance of a specific Kubernetes cluster. Or, use it to collect information from IoT devices or other non-traditional hardware. You can also employ Flex to combine data from an otherwise closed application with the performance data of the server running that app. Those are just a few use cases, but what about your specific case? Maybe someone has already created the one you need. 

The broader New Relic community has created a Github samples repository that lists over 200 open source solutions for custom data. Want to build your own? There’s an API for that. 

What do I need to know in order to set up Flex? 

You might be thinking “Sure, Flex can ingest my data. Does this require me to know some obscure language I have to code my custom solution in? I don't want to level up to become a sylvan elf soothsayer in a new coding language again!” 

Short story: no. It’s not like you have to learn regex or why list() and the [] operator in Python differ.  Yes, Flex requires you to query your data so you need to know a little bit of New Relic Query Language or NRQL (yes, NRQL is also the name of our Director of Product Management’s cat). You’ll also need to know some YAML, which is a human-readable, key-value pair type of config file.

We have both documentation on NRQL and a New Relic University course you can take, or you can just get up and running in under five minutes with a sample in the next parts of our Absolutely simple Flex blog series by my colleague and Principal Developer Relations Advocate, Leon Adato. He'll show you how to set up Flex and configure some metrics in just a few steps.