Accessing application logs in the context of application performance monitoring (APM) metrics has consistently been one of our most requested capabilities because it’s critical for efficiently troubleshooting performance issues. But until our new updated APM agents, developers had to manually request access to specific hosts and configure non-standard log forwarding to collect application logs separate from their APM metrics, traces, and events.
To address this developer pain point, when you update to our newest Java, .NET, Ruby, Python, Node.js, Go, and PHP APM agents, with compatible logging frameworks, you’ll automatically start collecting logs without the need to set up or configure anything.
APM and logging
Before we dive in, let’s clarify the relationship between logging and APM. Logging means recording events or messages an application generates in a log file. Then you can analyze those logs to diagnose and resolve issues with the app. Logging provides a detailed record of what happened in an application and is helpful for debugging.
APM is a more comprehensive approach to monitoring applications, going beyond logging. APM tools monitor and measure the performance of applications in real time, providing insights into application behavior, code execution, database queries, network latency, and other metrics that impact application performance. APM tools provide a holistic view of application performance so you can quickly identify the root cause of performance issues. Combining logs and APM is powerful—logs in context of APM data make it easier to find the root cause of issues.
Benefits of our updated APM agents
With the updated APM agents, you get granular configurations tailored for your billing and security needs and the option to opt-out anytime. When you update your agent, you get these three key benefits:
- Eliminate manual log configuration and maintenance necessary to collect and forward logs to New Relic without a dedicated log forwarder or lengthy manual configurations
- Minimize context-switching by viewing application logs in context with APM metrics, traces, and events.
- Troubleshoot issues faster by accessing logs inside the APM with enhanced UI that surface the logs you need alongside other relevant telemetry data.
Security, compliance, and control
When collecting logs, it is critical to ensure security, compliance, and control. This is why we are including robust support for the following:
- Opt-out anytime: Turn off automatic forwarding at any point by configuring your agent or using the New Relic data management hub.
- Ingest control: Use in-agent log sampling to manage ingested volume and avoid duplicating ingest and get 3X more value than alternate log management solutions.
- Compliance: Log collection is disabled by default for HIPAA-enabled accounts, and accounts where high security mode is in use, even after you upgrade the agent.
- Data security: Prevent sending PII, PHI, or any other sensitive data via customizable security configurations.
Note: Application logs in APM are only available after an APM agent update. Until you upgrade, you won't see changes in your account.
Let’s find out more about these new features.
Collect logs via the APM agent
Logs are an integral part of troubleshooting application performance and health, but collecting application logs is a complex and often frustrating experience that requires domain expertise and access to your underlying environment.
Now, when you upgrade your APM agents (for supported application languages), they will automatically collect and forward logs to New Relic by default, without the need to set up or configure anything. No more need for container sidecars or third-party applications1. In a single deployment, you can automatically monitor both applications and logs.
Not every language or logging framework is currently supported. The APM agents for the languages listed here now support automatic logs in context:
Learn more about the APM agent log configuration.
View application logs in context
Logs are better with context. By adding metadata to your logs, New Relic provides context that links your telemetry event types and entities—like APM—across our platform. Having application context in a log means that you can see all logs related to an application in a single view. You can troubleshoot and analyze your data without the need to switch between screens or tools.
Use logs where you need them in the enhanced UI
We’ve improved our user interface to surface the logs you need alongside other relevant telemetry data. You get detailed logs across the New Relic platform, including errors inbox, application traces, and more. By combining APM and log data seamlessly, it's easier than ever to troubleshoot and identify the root cause of issues.
Ensure data privacy by managing your logs
Data collection always carries the risk of inadvertent sensitive data disclosure. Logs can contain sensitive data that should not leave your environment, regardless of how you’re sending those logs. You always have direct control over your systems and which data you send to New Relic. Be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending PII, PHI, or any other sensitive data.
For more information, see our security documentation for log management.
Opt out and turn automatic log forwarding off
You may not want to ingest logs based on your organization’s security guidelines or to control costs. You can turn off logs by configuring your agent config file locally, or you can disable it at the account level in the data management hub.
APM agent log ingestion is disabled by default at the account level for HIPAA-enabled accounts, and accounts where high security mode is in use, even after you upgrade the agent.
You can also enable log ingest from APM agents at any time by using the self-service toggle in the New Relic data management UI. Learn more about how to toggle log ingestion on or off.
Get started with application logs in context
To begin using automatic logs in context, log in to your New Relic account or sign up for a free account today. Make sure you are using the latest supported APM agent and start using the power of automatic logs in context. Your free account offers 100 GB/month of data ingest, one full-platform user, and unlimited basic users.
1 Third-party log forwarders might be required in special circumstances.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.