View logs in context of APM data without any manual configuration

Accessing application logs in the context of application performance monitoring (APM) metrics has consistently been one of our most requested capabilities because it’s critical for efficiently troubleshooting performance issues. But until our new updated APM agents, developers had to manually request access to specific hosts and configure non-standard log forwarding to collect application logs separate from their APM metrics, traces, and events.

To address this developer pain point, when you update to our newest Java, .NET, Ruby, Python, Node.js, Go, and PHP APM agents, with compatible logging frameworks, you’ll automatically start collecting logs without the need to set up or configure anything.

APM and logging

Before we dive in, let’s clarify the relationship between logging and APM. Logging means recording events or messages an application generates in a log file. Then you can analyze those logs to diagnose and resolve issues with the app. Logging provides a detailed record of what happened in an application and is helpful for debugging.

APM is a more comprehensive approach to monitoring applications, going beyond logging. APM tools monitor and measure the performance of applications in real time, providing insights into application behavior, code execution, database queries, network latency, and other metrics that impact application performance. APM services provide a holistic view of application performance so you can quickly identify the root cause of performance issues. Combining logs and APM is powerful—logs in context of APM data make it easier to find the root cause of issues. 

Benefits of our updated APM agents

With the updated APM agents, you get granular configurations tailored for your billing and security needs and the option to opt-out anytime. When you update your agent, you get these three key benefits: 

• Eliminate manual log configuration and maintenance necessary to collect and forward logs to New Relic without a dedicated log forwarder or lengthy manual configurations
• Minimize context-switching by viewing application logs in context with APM metrics, traces, and events.
• Troubleshoot issues faster by accessing logs inside the APM with enhanced UI that surface the logs you need alongside other relevant telemetry data.

Security, compliance, and control

When collecting logs, it is critical to ensure security, compliance, and control. This is why we are including robust support for the following: 

• Opt-out anytime: Turn off automatic forwarding at any point by configuring your agent or using the New Relic data management hub. 
• Ingest control: Use in-agent log sampling to manage ingested volume and avoid duplicating ingest and get 3X more value than alternate log management solutions. 
• Compliance: Log collection is disabled by default for HIPAA-enabled accounts, and accounts where high security mode is in use, even after you upgrade the agent.
• Data security:  Prevent sending PII, PHI, or any other sensitive data via customizable security configurations.

Note: Application logs in APM are only available after an APM agent update. Until you upgrade, you won't see changes in your account.

Let’s find out more about these new features.

Collect logs via the APM agent

Logs are an integral part of troubleshooting application performance and health, but collecting application logs is a complex and often frustrating experience that requires domain expertise and access to your underlying environment.

Now, when you upgrade your APM agents (for supported application languages), they will automatically collect and forward logs to New Relic by default, without the need to set up or configure anything. No more need for container sidecars or third-party applications¹. In a single deployment, you can automatically monitor both applications and logs.

Not every language or logging framework is currently supported. The APM agents for the languages listed here now support automatic logs in context:

Learn more about the APM agent log configuration.

View application logs in context

Logs are better with context. By adding metadata to your logs, New Relic provides context that links your telemetry event types and entities—like APM—across our platform. Having application context in a log means that you can see all logs related to an application in a single view. You can troubleshoot and analyze your data without the need to switch between screens or tools.

Use logs where you need them in the enhanced UI

We’ve improved our user interface to surface the logs you need alongside other relevant telemetry data. You get detailed logs across the New Relic platform, including errors inbox, application traces, and more. By combining APM and log data seamlessly, it's easier than ever to troubleshoot and identify the root cause of issues.

Ensure data privacy by managing your logs

Data collection always carries the risk of inadvertent sensitive data disclosure. Logs can contain sensitive data that should not leave your environment, regardless of how you’re sending those logs.  You always have direct control over your systems and which data you send to New Relic. Be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending PII, PHI, or any other sensitive data.

For more information, see our security documentation for log management.

Opt out and turn automatic log forwarding off

You may not want to ingest logs based on your organization’s security guidelines or to control costs. You can turn off logs by configuring your agent config file locally, or you can disable it at the account level in the data management hub.

APM agent log ingestion is disabled by default at the account level for HIPAA-enabled accounts, and accounts where high security mode is in use, even after you upgrade the agent.

You can also enable log ingest from APM agents at any time by using the self-service toggle in the New Relic data management UI. Learn more about how to toggle log ingestion on or off.

¹ Third-party log forwarders might be required in special circumstances.