10 New Cloud Integrations for AWS Monitoring with New Relic Infrastructure

To achieve observability of your Amazon Web Services (AWS) infrastructure, you need a connected view that accounts for all entities spread across your AWS hosts and services. As an AWS partner, we understand that your cloud usage is ever-evolving—and AWS offers a wealth of services that allow easy access to core application requirements such as storage, networking, message queues, data streaming, and firewalls.

In the run up to AWS re:Invent 2019, the New Relic Cloud Monitoring team has been hard at work. This year, we’re excited to announce 10 new cloud integrations for AWS, adding to an already significant list of powerful integrations for the New Relic platform.

These AWS integrations gather data using the AWS CloudWatch API, and each integration comes with a set of New Relic One entities and a load of metrics. Be sure to check out the documentation page for each integration to see the complete list of metrics and entities.

To use these—or any New Relic AWS integration—you’ll need a New Relic Infrastructure Pro account, and you’ll need to connect your Amazon account to New Relic Infrastructure.

With that said, let’s take a look at the integrations.

The latest AWS cloud integrations from New Relic

Amazon MQ

The Amazon MQ service helps you manage the setup and operation of Apache ActiveMQ message brokers in the AWS cloud.

Metric data collected by the integration includes:

• CurrentConnectionsCount: The current number of active connections on the current broker
• HeapUsage: The percentage of the ActiveMQ JVM memory limit that the broker currently uses
• CpuUtilization: The percentage of allocated EC2 compute units that the broker currently uses

For more see the New Relic documentation.

Amazon Managed Streaming for Apache Kafka (MSK)

Apache Kafka is a streaming data processing tool. Amazon Managed Streaming for Apache Kafka (MSK) abstracts away the management of Kafka so you don’t have to worry about maintaining a data streaming pipeline.

Metric data collected by the integration includes:

• GlobalPartitionCount: Total number of partitions across all brokers in the cluster
• LeaderCount: The number of leader replicas
• ProduceMessageConversionsPerSec: The number of produce message conversions per second for the broker

For more see the New Relic Documentation. Amazon MSK exposes metrics in a Prometheus compatible format. You can also use New Relic Prometheus OpenMetrics integration to monitor MSK.

AWS Glue

AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy for customers to run jobs that prepare and load their data in the AWS Glue Data Catalog.

Metric data collected by the integration includes:

• driver.aggregate.bytesRead: The number of bytes read from all data sources by all completed Spark tasks running in all executors
• driver.aggregate.numFailedTasks: The number of failed tasks
• driver.aggregate.recordsRead: The number of records read from all data sources by all completed Spark tasks running in all executors

For more see the New Relic Documentation.

Amazon Athena

Amazon Athena is a serverless interactive query service that makes it easy to run standard SQL queries against data stored in Amazon S3, paying only for the queries you run.

Metric data collected by the integration includes:

• ProcessedBytes: The amount of data in MB that Athena scanned per query
• TotalExecutionTime: The amount of time in seconds it takes Athena to run the query

For more see the New Relic Documentation.

AWS Step Functions

Use AWS Step Functions to create serverless workflows from multiple AWS services. This integration collects all kinds of data about Step Functions, including activity data, LambdaFunction data, ServiceIntegration data, and ApiUsage data.

Metric data collected by the integration includes:

• ExecutionTime: The interval, in milliseconds, between the time the execution starts and the time it closes
• ActivityRunTime: The interval, in milliseconds, between the time the activity starts and the time it closes
• LambdaFunctionRunTime: The interval, in milliseconds, between the time the Lambda function starts and the time it closes

For more see the New Relic Documentation.

AWS Direct Connect

A private connection between AWS and your datacenter or office is essential. With AWS Direct Connect, you’ll reduce network costs, increase bandwidth throughput, and have more robust networking than with Internet-based connections.

Metric data collected by the integration includes:

• connectionState: The state of the connection: 0 indicates DOWN and 1 indicates UP
• connectionLightLevelTx: Indicates the health of the fiber connection for egress (outbound) traffic from the AWS side of the connection
• connectionPpsEgress: The packet rate for outbound data from the AWS side of the connection

For more see the New Relic Documentation.

Amazon DocumentDB (with MongoDB compatibility)

As a fully managed document database service, Amazon DocumentDB supports MongoDB workloads. It’s fast, scalable, and highly available. Data collected by the New Relic integration is broken out between DocumentDB Cluster data, DocumentDB ClusterByRole data, and DocumentDB Instance data.

Metric data collected by the integration includes:

• ReadIOPS: The average number of disk read I/O operations per second; Amazon DocumentDB reports read and write IOPS separately, and on one-minute intervals
• BackupRetentionPeriodStorageUsed: The total amount of backup storage in GB used to support the point-in-time restore feature within the Amazon DocumentDB's retention window
• WriteThroughput: The average number of bytes written to disk per second

For more see the New Relic Documentation.

AWS WAF

Protect your web applications and keep them secure with AWS WAF, a managed firewall.

Metric data collected by the integration includes:

• AllowedRequests: The number of allowed web requests
• BlockedRequests: The number of blocked web requests
• PassedRequests: Passed requests are requests that don't match any rules contained in the rule group

For more see the New Relic Documentation.

AWS AppSync

AWS AppSync allows app developers to use GraphQL to create flexible APIs to securely access, manipulate, and combine data from one or more data sources. The New Relic integration provides data about 400 and 500-level error counts, as well as latency.

Metric data collected by the integration includes:

• 4XXError: The number of errors captured as a result of invalid requests due to incorrect client configuration
• 5XXError: Errors encountered during the execution of a GraphQL query
• Latency: The time between when AWS AppSync receives a request from a client and when it returns a response to the client

For more see the New Relic Documentation.

AWS QLDB

Ledger applications can be challenging to build and manage, but as a fully managed ledger database, AWS QLDB provides a transparent, immutable, and cryptographically verifiable transaction log owned by a central trusted authority.

Metric data collected by the integration includes:

• journalStorage: The total amount of disk space used by the ledger's journal, reported in 15-minute intervals
• readIOs:The number of disk read I/O operations, reported in one-minute intervals
• indexedStorage: The total amount of disk space used by the ledger's tables, indexes, and indexed history, reported in 15-minute intervals

For more see the New Relic Documentation.

New Relic sets you up for AWS success

If you seek to modernize your business in the AWS cloud by taking advantage of the best services AWS has to offer, know that the New Relic platform provides essential observability you need to move fast with confidence—even when troubleshooting the most modern AWS services.

But our AWS integrations don’t operate in isolation; they’re built with the full application stack in mind. When used in conjunction with New Relic EKS monitoring or New Relic monitoring for AWS Lambda, you’ll gain the full speed, scalability, flexibility, and resiliency of operating in the cloud. Additionally, our guide to optimizing your cloud native environment explains how to manage availability and performance by using an instrumentation and data-oriented approach to rightsize instances, fine-tune databases, modify storage usage, and better configure essential services in your application stack.

These capabilities are just the tip of the iceberg! Check out our full list of integrations, start a trial of New Relic Infrastructure, and get visibility into your AWS environment today.