How to achieve CDN observability with Amazon CloudFront logs

Monitoring the performance of your content delivery network (CDN) is critical, and logs can provide essential performance metrics for in-depth analysis. CDN monitoring allows you to proactively detect and troubleshoot issues. By monitoring key metrics such as the number of cache hits/misses and HTTP response codes for cached objects, and receiving alerts for anomalies or errors, you can quickly identify and address issues before they affect end-users. This helps minimize downtime, resolve performance problems, and maintain a positive user experience.

To monitor and analyze the performance of your CDN using Amazon CloudFront, you can use real-time logs to get detailed information about every request that CloudFront receives. Real-time logs are delivered to a data stream and can be forwarded to your favorite observability platform, New Relic, where you can use the power of log patterns, log filtering with selected properties, and alerts.

This guide will show you how to set up real-time log forwarding for Amazon CloudFront CDN. You will use access logs of a CloudFront CDN that's hosting a simple single-page application. You’ll then forward the access logs using the Amazon Kinesis Data Stream and the Kinesis Data Firehose delivery stream. This will enable you to create real-time alerts and live dashboards, investigate anomalies, or quickly respond to deteriorating events.

Prerequisites

To successfully follow this guide, you’ll need these prerequisites:

1. An active account on Amazon Web Services.
2. A basic understanding of Amazon Kinesis Data Streams.
3. A New Relic account. If you don't have one yet, sign up for free. Your account includes up to 100 GB/month of free data ingest, one full-platform user, and unlimited basic users.
4. New Relic ingest license key. You can get your license key from https://one.newrelic.com/api-keys.

How to forward CloudFront logs to New Relic

You’ll follow three steps to forward Amazon CloudFront logs to New Relic:

1. Create a Kinesis Data Stream for your CloudFront logs.
2. Create a delivery stream with Kinesis Data Firehose to consume and forward log data to New Relic.
3. Enable real-time logging for CloudFront and connect to the Kinesis Data Stream.

Step 1: Create the Kinesis Data Stream

Start by logging into your AWS account, and then navigate to services and select Amazon Kinesis. Within the Amazon Kinesis service, locate and select Data streams, then select the Create data stream button to start setting up your data stream. You’ll see the same configuration screen that’s shown in the next image. 

Within the form, give your data stream a name and choose the data stream capacity mode that suits your requirements. The capacity mode you select will depend on factors such as the volume of your logs and the desired throughput. 

When you’ve finished filling out the form, select the Create data stream button.

Step 2: Configure the delivery stream

To deliver data from your Kinesis Data Stream to the New Relic platform, you need to set up a delivery stream. Watch the next video, or read the following instructions:

1. With the Kinesis Data Steam you just created, locate the Consumers section and select the Process with the delivery stream option. This will open a configuration screen to create a Kinesis Data Firehose delivery stream that will consume the data from the Kinesis Data Stream and deliver it to the New Relic platform.
2. Within the Choose source and destination section, for Destination, search for and select New Relic.
3. Within the Delivery stream name section, input a name for your delivery stream.
4. Within the Destination settings section:
   1.  Specify the appropriate HTTP endpoint URL for New Relic logs. You can choose either New Relic logs - US (https://aws-api.newrelic.com/firehose/v1) or New Relic logs - EU (https://aws-api.eu.newrelic.com/firehose/v1) based on your location.
   2. For API key, input your ingest license key. You can find and copy your ingest license key in New Relic at https://one.newrelic.com/api-keys.
   3. Select Add parameter to create a new parameter with Key as logtype and Value as CDN-rtl. This will help to filter down the logs in New Relic.
5. Within Backup settings, you can configure an S3 bucket as a backup destination for the Kinesis Data Firehose delivery stream in case of delivery failures. Choose between backing up All data or Only failed data according to your preferences.

Step 3: Enable real-time logging for CloudFront

To create a real-time configuration for CloudFront logs and associate it with the data stream you created, watch the next video, or read the following instructions:

1. Go to the CloudFront service and select the Logs option under the Telemetry section. 
2. Select the Real-time configurations tab, then select Create configuration. This will open up a new form.
3. Within the configuration form, you’ll need to fill out five fields:

• Name: Simply give a name for this real-time logging configuration.
• Sampling rate: This determines the percentage of requests that will be sent to Kinesis Data Streams as real-time log records. The value can be between 1-100%. For this example, set it to 100%.
• Fields for the logs: choose which fields to include in your real-time logs, consider your requirements and the type of analysis you plan to perform. For simplicity, select all fields.
• Endpoint for data stream: Choose the Kinesis Data Stream you created earlier.
• Distributions and cache: Choose the specific CloudFront distributions and caches that you want to monitor.

4. When you are done filling out the form, select Create configuration.

New Relic log parsing

CDN logs that flow in are unstructured, unformatted, and difficult to make sense of without any associated identifiers. This is the default log format that CloudFront writes in.

This is where New Relic logs UI capabilities come to the rescue. You’re going to use the log parser capability in the logs UI and apply a Grok pattern to enrich these logs.

Here’s how to create a new parsing rule in the New Relic logs UI:

1. Open New Relic.
2. Select Logs from the left-most menu.
3. Select Parsing from the Logs menu.
4. Select Create parsing rule, which will open a new form.
5. Within the form, specify the following values:
   1. Name: name your parsing rule.
   2. Filter logs based on NRQL: input the parameter key (logtype) and value (the parameter value CDN-rtl that you added earlier while creating Kinesis Firehose).
   3. In the Parsing rule and output section, you can use the query logtype='CDN-rtl' in the "Filter logs based on NRQL" field. Alternatively, you can paste a log sample in the "Choose your matching log" section and add the following pattern in the "Parsing rule" section. This Grok pattern assumes that you have selected all fields for your real-time logs and parses them by identifying the spaces and adding individual identifiers to your log fields.

When you are finished filling out the form, select Create rule.

Once your parsing rule takes effect, all new incoming logs will be parsed and formatted according to the specified Grok pattern. Here's what your logs will look like now:

Now not only can you view parsed results on the UI, but you can also query your logs with NRQL using specific attributes from the logs. For example, you can query just on one property `x_cache_status` as seen in the screenshot below