Terraform is a powerful tool that helps you define and manage your cloud and on-prem resources, but it can be challenging to work with HCL (Hashicorp Configuration Language), especially when it comes to creating dynamically generated dashboards. Part one of this series shows how to create and maintain a New Relic dashboard in Terraform using JSON templates. Now you’ll learn how to build a complex dashboard from simple input data and then automate the generation of data to allow the dashboard to automatically update itself. In part three of this series, Using Terraform to generate New Relic dashboards from NRQL queries, you'll learn how to use Terraform with NRQL queries to generate a dynamic New Relic dashboard based on the result of a query.
For this example, you’ll create a dashboard for showing various metrics using global API data that's available for free in every account. You’ll use NRQL queries to power dashboards that show data regarding API calls made to Amazon Web Services, Google, Microsoft, and Facebook. The dashboard will contain a row of data widgets for each API service that includes total API calls, throughput, and latency. When you’re done, you’ll have a dynamically created dashboard that looks like this:
Normally, it can be tedious to build out the same set of widgets multiple times, but with template expressions, you can generate the dashboard without repeating the configuration details for each row of widgets.
Prerequisites
Note: To create New Relic dashboards in Terraform using JSON templates, you need to be using provider version 3.4.0 or above.
To follow along with the sample code in this tutorial, you’ll need to have Terraform installed. You can find instructions for installing Terraform and guidance on how to run the examples in nr-terraform-json-dashboard-examples.
1. Build the dashboard layout
The first task is to build out what a single row of widgets should look like. The finished example includes the following:
- Full-width Markdown text row displaying the name of the API
- Large billboard widget showing total API calls
- Smaller billboards with additional metrics
- Two time series charts
- Customized widgets, including custom colors in the time series charts
The next image shows what a row in the dashboard looks like.
If you would like to create your own dashboard then go ahead and build out a row of widgets. Otherwise, if you want to follow along with this example you can use the example dashboard template from the sample code here: dashboards/composed_widgets.json.src
.
Once you have built the first row of the dashboard, copy the JSON to the clipboard (or use the sample code) and save it to a file called composed_widgets.json.tftpl
in the dashboards folder. If you inspect the file, you should see that your row of widgets is contained in the widgets
array:
{
"name": "Dashboard Name",
"description": null,
"permissions": "PUBLIC_READ_ONLY",
"pages": [
{
"name": "Page Name",
"description": null,
"widgets": [ …YOUR-WIDGETS-HERE… ]
}
]
}
2. Define input configuration
Next, you need to configure your inputs. The first step is to define a data type to hold a list of services you want to display in your dashboard. You’ll add this to the top of the dash_composed.tf
file:
variable "config" {
type = list(object({
name = string
domain = string
}))
}
Next, create a list of services in a terraform.tfvars
file. It’s a best practice to put your variables into a separate file. Doing so makes it easier to review and update them. Here you provide the values for the data type that was defined in the previous code snippet:
config = [
{
name = "Amazon Web Services"
domain = "amazonaws.com"
},
{
name = "Google APIs"
domain = "googleapis.com"
},
{
name = "Microsoft"
domain = "microsoft.com"
},
{
name = "facebook.com"
domain = "facebook.com"
},
]
The next step is to dynamically generate a row of widgets for each service in this list.
3. Generate the dashboard widgets
You’ll need to combine the configuration from your terraform.tfvars
with the dashboard template file in dash_composed.tf
. You should use the templatefile()
function as shown in the next code sample. Store it in a local variable so you can easily refer to it. Here's the code in dash_composed.tf
:
locals {
template_render = templatefile(
"${path.module}/dashboards/composed_widgets.json.tftpl",
{
ACCOUNTID = var.accountId
CONFIG = var.config
}
)
}
Note that CONFIG
is a reference to the configuration object that holds the list of API domains and names we defined earlier. The next step is to iterate over the list of CONFIG
items, creating a dashboard row for each one. You’ll do this within the JSON template itself using template syntax like this:
[..outer container code here…]
%{~ for index, api in CONFIG ~}
%{ if index!=0 },
%{ endif }
[WIDGET CODE HERE]
%{ endfor ~}
[...Outer container code continues here...]
This code iterates over the CONFIG
object. For each iteration, two variables are made available:
index
: This is the index of the item, which you'll use to terminate the loop after iterating is finished. The JSON array needs to be valid with no trailing comma!api
: This contains the current config item we are iterating, which includes the key-value pairs fordomain
andname
. You’ll useapi.domain
andapi.name
to populate dashboard values.
Next, you need to dynamically generate rows for your New Relic dashboard. New Relic dashboards are arranged in a grid of columns and rows. Each widget in a dashboard has a column and row value that defines where it appears on the page. However, if you set this manually for every widget, you’ll have a lot of repeated code. Instead, let’s make this dynamic so no repeated code is needed.
Each service will have metrics in a layout that is four rows high. The first row contains the name of the service. The second through fourth rows contain the metrics. Here’s an example:
You can display this dynamically using the index of the item. The first item has an index of 0, so it will occupy rows one to four. Then, since each item needs to have four rows, you just need to multiply the index by four with this formula: (index * 4)
. So the first row of each item should be (index * 4) + 1
, the second row should be (index * 4) + 2
, and so on.
Here’s an example of the layout for the title widget:
"layout": {
"column": 1,
"row": ${(index * 4) + 1 },
"width": 12,
"height": 1
},
See the example template file for more information on how each row is dynamically created, as well as details on the static column layout for each widget.
Now that the layout is dynamic, it’s time to dynamically generate the metrics for each service, too. Here’s an example NRQL query that the dashboard will use:
"query": "SELECT count(*) as 'API Calls (last hour)' from Public_APICall where api='amazonaws.com' since 1 hour ago"
This query works, but it’s hardcoded to “amazonaws.com”
. To make it dynamic, you can use string interpolation to replace the hardcoded value with the value from the CONFIG
object:
"query": "SELECT count(*) as 'API Calls (last hour)' from Public_APICall where api='${api.domain}' since 1 hour ago"
Note that api='amazonaws.com'
has been updated to api='${api.domain}'
. Now the query is reusable, and with each iteration through the loop, the correct NRQL query will be made for each domain name. Refer to the example in the template file.
You also need to change the content of the Markdown widget so that it displays the API name. Instead of being hardcoded like this:
"text": "# Amazon Web Services (amazonaws.com)"
Once again, you’ll use string interpolation to make it dynamic:
"text": "# ${api.name} (${api.domain})"
You can see this changed here in the example template file.
4. Create the dashboard resource
Next, add the Terraform resource to create the dashboard using the rendered output of your template file. The following code is in dash_composed.tf:
resource "newrelic_one_dashboard_json" "composed_dashboard" {
json = local.template_render
}
resource "newrelic_entity_tags" "composed_dashboard" {
guid = newrelic_one_dashboard_json.composed_dashboard.guid
tag {
key = "terraform"
values = [true]
}
}
output "composed_dashboard" {
value = newrelic_one_dashboard_json.composed_dashboard.permalink
}
Next, you can apply the Terraform configuration. Your dashboard should show rows of widgets for each API in your configuration file.
5. Debugging configuration errors
If there’s an error and dashboard creation fails, it’s probably because the templatefile
produced invalid JSON content. Trailing or missing commas are especially worth watching out for. You can debug this by outputting the rendered template file to your screen. You can then verify it and find that missing comma.
To do so, add the following to dash_composed.tf view to the rendered templatefile
:
output "composed_dashboard_src" {
value = local.template_render
}
Now that you know how to build dynamic dashboards from a simple configuration in New Relic with Terraform, you’ll find that it’s much easier to quickly create new dashboards with simple inputs. However, what if you wanted to generate a dashboard that shows you the top five APIs? That’s a little different because the rows will always be changing. In part three, you’ll learn how to dynamically generate dashboards based on changing source data.
Next steps
- Read part one of the series, Creating dashboards with Terraform and JSON templates, to learn how to quickly create New Relic dashboards using Terraform and JSON templates.
- Read part three of the series, Using Terraform to generate New Relic dashboards from NRQL queries, to learn how to use Terraform with NRQL queries to generate a dynamic New Relic dashboard based on the result of a query.
If you’re not using New Relic yet, get started with New Relic for free. Your free account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.