Leverage Our Lambda Runtime Logs API Integration for Better Serverless Observability

Engineers working with serverless cloud-based architectures report “‘debugging”’ and other observability challenges as their top concerns, over cost controls and understanding architectural complexity. While serverless has eliminated many of the operations problems engineers faced in the past, the challenge of effectively monitoring and understanding issues remains significant, especially in a system where it’s easy for small teams to deliver production microservices.

Today, these challenges are lessened with Amazon Web Services (AWS) announcing the release of the Lambda Runtime Logs API, which allows extensions to access logging data from AWS Lambdas and the Lambda execution environment. Concurrently, New Relic is releasing updates to support the Logs API in our Lambda monitoring toolkit.

The Logs API allows extensions to subscribe to three different logs streams:

1. Function logs, which are generated by the Lambda function and written to or emitted as messages
2. Platform logs generated by the Lambda platform (such as the START, END, and REPORT lines), and
3. Extension logs, which are produced by extension code

The New Relic extension will use the Logs API to bring feature parity with a solution that monitors Amazon CloudWatch for logged events. The result is a cleaner install path to allow New Relic to monitor serverless compute applications.

There are multiple paths to implementation using AWS Lambda’s extensions and the Logs API. New Relic’s implementation of the integration uses the execution time available during function execution to improve the time data takes to reach your dashboard. New Relic’s approach is highly optimized for platform logs, ingesting data on your Lambda’s performance with minimal latency and cost.

In this early phase, there are some AWS Lambda and New Relic caveats to be aware of. Currently, New Relic’s integration with the Lambda Runtime Logs API will only send data pertinent to New Relic’s observability dashboards and can’t be configured to send all logs. To ingest more logs, users should install the New Relic log ingestion Lambda, which will send logs from CloudWatch. Users should also be aware that using the Logs API will not prevent logs from going to CloudWatch by default, and those who want to prevent their Lambdas from logging to CloudWatch will need to remove the IAM permission from their Lambdas.

New Relic's ongoing collaboration with AWS brings real insight into the functioning of your AWS Lambda-based applications. Our collaboration with AWS has continued to grow in 2020, from our implementation of the Extensions API, to advanced infrastructure monitoring using the CloudWatch API and adopting the OpenTelemetry standard.

Sign up for a free New Relic One account to start functioning faster today.