Monitoring PM2 in production

In large-scale Node.js production environments, monitoring multiple applications can become challenging. The New Relic APM agent for Node.js helps capture logs, traces, and in-depth performance metrics from individual applications. But what about the overall health and resource consumption of all Node.js processes themselves, and critical process-level metrics like CPU and memory usage?

PM2 is a popular process manager for Node.js applications, designed to simplify deployment and ensure reliability. It provides robust features, including automatic application restarts, load balancing, and monitoring capabilities, making it an essential tool for managing production-grade Node.js environments.

PM2's monitoring API provides easy access to detailed telemetry data, like active processes, and resource consumption, along with additional helpful details like Git commit, active branch, Node.js version, and entry script. These insights can be invaluable when troubleshooting and identifying the root cause of performance issues.

What can I monitor from PM2?

PM2 API provides generous information that can be captured. Ideally, all the metrics that can be visualized using the PM2’s monitor API on your host can be captured and exported to New Relic.

In this blog post, we’ll focus on several key metrics that are important for monitoring PM2. 

Application identity and host information

Application details, including the host ID, for applications currently running as part of PM2, make it easy to pinpoint which instance is associated with each application. This is especially helpful in environments with multiple services, as you can easily identify specific processes and manage them effectively.

Metrics: appName, hostId

Resource metrics

Metrics from each process, like CPU and memory usage, provide essential insights into the resource consumption and performance of your application. With axm_monitor, you can also track HTTP latency, active requests, and event loop latency to monitor responsiveness in real time.

Metrics: monit.cpu, monit.memory, axm_monitor (e.g., HTTP Latency, Active Requests, Event Loop Latency)

Process information

Process-level details such as process ID, status, and uptime help monitor each application's health and lifecycle. Tracking when a process was last updated or started helps you identify issues like frequent restarts or unusually high uptime.

Metrics: pid, pm2_env.status, pm2_env.pm_uptime, pm2_env.created_at, timestamp, pm2_env.update_time

Debug pointers: Logs, errors, restarts, and crashes

Debugging details like log paths, error codes, and restart counts allow quick troubleshooting. Having both the standard output and error logs, alongside information on restarts, helps pinpoint stability issues or potential bugs within applications.

Metrics: pm2_env.error_file, pm2_env.out_file, pm2_env.exit_code, pm2_env.unstable_restarts, pm2_env.restart_time, pm2_env.status

Version control and deployment insights

Deployment details, including Git commit history and Node.js version, make it easy to track deployed versions. PM2 captures branch, revision, and commit data, giving clear visibility into which version is running and simplifying root cause analysis after code changes.

Metrics: pm2_env.node_version, pm2_env.version, pm2_env.versioning.*

How do I capture insights from PM2? 

Currently, there’s no direct integration between New Relic agents and PM2, but we can build our own integration using Flex. Flex is an easy and agentless option to build integrations between your data source and New Relic. 

Configuration 

Create a new configuration file named pm2_monit.yml, then add the following configuration.

This configuration is simply calling the PM2 jlist API and sanitizing the JSON output data using JQ. JQ is a command line utility that's used to process, query, and transform JSON data. Flex has support for JQ built in, which makes sanitizing and transforming the data easier. 

As the original output can also include additional, sensitive data, which could be unnecessary, it can be easily dropped with the JQ and remove_keys functions of Flex.

Basic configuration

JSON transformation with JQ

At this stage, the output from pm2 jlist is a raw JSON. Let's sanitise and transform the JSON output with JQ to retain only the necessary fields, using the remove_keys and rename_keys functions to streamline the data further:

Switching user context

The command block in Flex configuration switches the user context to where PM2 processes are running, which is necessary, as Flex runs under the root (sudo) user by default. If you're running PM2 locally on Windows or Mac, use npx pm2 jlist command block instead of the Linux-specific command.

The complete configuration should look like this:

NRQL provides an easy way to query and visualize metrics, with powerful dashboard capabilities. Below is an example of a custom dashboard setup specifically designed for PM2 monitoring. It captures various metrics, including memory usage per application, CPU vs. memory for all active applications, and the latest app revision details of the active processes with PM2.