New Relic and Trend Micro Cloud One: Better Together

New Relic has partnered with Trend Micro to help our Amazon Web Services (AWS), Microsoft Azure, and Google Cloud customers further strengthen their cloud observability and security posture. New Relic’s Conformity integration automatically brings in all your cloud infrastructure configuration data from Trend Micro Cloud One - Conformity into New Relic for all your observability needs. With this partnership, you can build a complete cloud visibility strategy by complementing New Relic-powered observability with added visibility into your cloud security and compliance posture.

About Cloud One and Conformity

Trend Micro Cloud One is a security services platform for cloud developers that delivers the broadest and deepest cloud security offering in one solution, enabling you to secure your hybrid and multi-cloud security infrastructure with clarity and simplicity. Cloud One consists of seven cloud security services that address workload security, container security, file storage security, network security, application security, open source security, and security posture management.

Cloud One - Conformity provides cloud best practices to help cloud developers fulfill their side of the shared responsibility model with continuous guardrails for building well in the cloud. Because industry standards and compliance requirements constantly change, it's important to know that your workloads are automatically evaluated to check alignment with industry best practices. These continuous scans for compliance and industry standards provide actionable intelligence to know where and how to improve your security posture. They serve as a roadmap for building secure, efficient, and automated cloud operations.

Conformity’s cloud best practices align you with Azure and AWS Well-Architected frameworks and insights about how well aligned you are to common compliance standards and frameworks for your workloads running in the cloud. Conformity auto-checks using nearly 1,000 cloud service configuration best practices across more than 90 services from AWS, Microsoft Azure, and Google Cloud Platform. You can easily remedy any misconfigurations by using the Conformity Knowledge Base, which is a continually growing library that contains cloud service configuration guides. Conformity runs a bot that scans your cloud resources, compares them to the rules, and then generates checks as a result of the scans. The Compliance and Conformity Report in Conformity supports more than a dozen standards including AWS Well-Architected, HIPAA, GDPR, ISO 27001, SOC-2, and PCI-DSS. These reports give you an instant assessment of your organization’s cloud infrastructure compliance. You can take remediation measures to improve compliance levels, potentially avoiding reactive fixes and expenditures due to non-compliance.

From SIEM and SOAR to full-stack observability with New Relic 

Conformity users can enjoy the following benefits by using the integration with New Relic.

From noise to signal, with faceted dashboards, instantly and intuitively 

New Relic's Conformity integration helps your teams and various personas in your organization look at exactly the information you need to know and proactively identify potential problems before they become incidents. Teams can resolve any incidents that do occur in a timely and efficient manner. Tagging your AWS resources consistently and meticulously lets you obtain interesting insights into your security and compliance posture, related to teams, operating environments, applications, (and even business units, cost centers, and data classification).

Imagine that you notice a high-severity Conformity rule related to security that resulted in thousands of misconfigurations (failed checks). At first glance, this might seem alarming. But when you have the context of the application, the environment, or the team responsible for these failed checks, then you can make informed decisions. You'll know what to do with the situation, and if it needs attention at all. For example, the failed checks might be coming in from short-lived experimental applications that developers are running for rapid prototyping. Or, maybe the resources were launched by an automated software test that serves a specific purpose and isn't related to this particular security check. Otherwise, with so many instances of a failed check, it might take too long to get to the relevant resource that belongs to an Internet-facing production app with zero tolerance for a breach or outage.

You can filter your New Relic One dashboards by faceted attributes to build unique views of your Conformity data to gain instant insights into interesting use cases. For example, you can easily build a PCI compliance view just by adding one filter, as shown in the following example. You can build views that filter by AWS Region, or AWS Account. You can also filter by AWS Well-Architected categories like security and tags like environment, team, business unit, and cost center.

Instant troubleshooting with real-time correlation

Get real-time threat monitoring events from Conformity integrated with the rest of your existing workloads in New Relic, and other alerting and incident management tools you use for paging the operations teams. For faster troubleshooting, the events are routed to the right team with surrounding context. 

This enriched view gives you information related to AWS Well-Architected and other security and compliance information, next to telemetry for the workload you are viewing. You can quickly isolate problems with correlated events. For example, a spike in the cost of your application might be tied directly to the unused Amazon DynamoDB table or the underutilized Amazon Elastic Compute Cloud (Amazon EC2) instance. You can use a check report in Conformity to identify these issues.

New Relic Lookout: Applied Intelligence-powered visuals for proactive anomaly detection

New Relic’s Proactive Detection with Applied Intelligence powers the Lookout visualization, which instantly catches sudden and anomalous changes to your Conformity data. This proactive detection is available without additional setup. You can use your own custom queries that look at the attributes you use to facet and track anomalies. The following three example views illustrate how you can get immediate, proactive detection with intuitive visuals in New Relic Lookout:

New Relic’s Conformity integration

Let’s take a closer look at the integration so you understand its architecture, how to deploy it successfully, and the unique insights of using Conformity and New Relic together. The integration is open sourced in New Relic’s GitHub repository. It is packaged as an AWS solution using the serverless framework and deploys an AWS CloudFormation stack in your AWS account. Before using the integration, you must deploy the Conformity-to-S3 solution. The solution deploys all the AWS resources that are required to send Conformity checks and events in real-time to an Amazon Simple Storage Service (Amazon S3) bucket. This enables further integration with other services (in this case, New Relic).

The integration uses:

1. Amazon Simple Notification Service (Amazon SNS) communication channel to enable Conformity data to flow into your AWS account so it can eventually flow into New Relic. You can control which checks are sent to New Relic using filters when you set up your Amazon SNS channel in Conformity. Typically you might send all checks for a comprehensive view of your entire cloud environment, but there might be instances where you only care about failures or about a specific tag to limit the scope to your teams or business unit.
2. Conformity Public API to fetch all Conformity checks every 30 days. This is necessary in order to retain the entire catalog of Conformity checks, because New Relic purges all custom events older than 30 days by default. See New Relic Data Retention.
3. New Relic Event API to send Conformity checks (coming in from the Amazon SNS channel and the API) to New Relic as custom events. 

Architecture

The following architecture diagram shows the deployment view of the integration in your AWS account.

The integration deploys an AWS CloudFormation stack into your AWS account that contains an AWS Lambda function and an Amazon EventBridge rule. The Lambda function is triggered in real time whenever a new Conformity check (JSON file) is uploaded into the Amazon S3 bucket deployed by the Conformity-to-S3 solution. The Lambda function is also triggered once every 30 days, to pull all Conformity checks into New Relic. The Lambda function sends the checks into New Relic using the Python Telemetry SDK that uses New Relic Event API under the hood. The Event API requires access to the New Relic Insert API key. To fetch all Conformity checks, the Lambda function also needs access to the Conformity API key. Both the New Relic Insert API key and the Conformity API key are securely managed as AWS Secrets Manager secrets.

Deploying the integration

Before you begin, make sure the following requirements are met:

• You have access to an AWS account where you can deploy this solution. If you don't have an AWS account, you can sign up for the free tier.
• You have access to the Conformity service through the Trend Micro Cloud One 30-day free trial. Make sure you add your AWS, Azure, or Google Cloud account to Conformity. Note that you can add multiple cloud accounts to your Conformity console. If you don't already have a Cloud One account, you can sign up for a new account that includes a 30-day free trial.
• You have access to a New Relic account. Make sure you add your AWS account to New Relic. The account must be the same one that you deploy the integration to. Note that you can add multiple AWS accounts to your New Relic account. In this case, choose one account for deployment. If you don’t already have a New Relic account, you can sign up for a perpetually free account in the AWS Marketplace.

To prepare for deploying the integration, make sure dependencies are set up correctly:

• Deploy the Conformity-to-S3 solution to your AWS account. The integration depends on this deployment. The quickest way to deploy this solution is by using this CloudFormation template. Take note of the AWS Region you deployed the CloudFormation stack into. 
• Set up an Amazon SNS channel from your Conformity console to your AWS account, using the Amazon SNS topic that the Conformity-to-S3 solution deploys for you. You can find the topic’s ARN by looking up the TopicARN output from the CloudFormation stack where the Conformity-to-S3 solution was deployed.
• Add the necessary information to the config.dev.yml configuration file in the repository root directory. Replace the placeholders (in chevrons) with actual values. The file includes detailed comments with links for help with filling in the placeholders. If you want to fork this repository, do not commit this file, because it holds sensitive information about your Trend Micro Cloud One and New Relic One accounts.
• Set up AWS Command Line Interface (AWS CLI) on your machine or on your build host from where you plan to deploy the solution.
• Install the latest version of serverless CLI. Serverless is a Node.js CLI tool. If you don’t already have Node.js on your machine, you’ll need to install it first.
• The integration uses an AWS Lambda function using python packaged with serverless. Therefore, you need to have Docker installed, along with Python 3.
• Initialize the node modules by running this command from the project root directory:

Deployment 

Deploy the integration using serverless CLI by running the following command. If you have an AWS CLI profile set up, include –-profile <PROFILE_NAME>.

This deploys the solution using the configuration defined in config.dev.yml file in the us-east-1 AWS Region of your AWS account, by default. You must specify the AWS account the integration is deployed to, by setting up your AWS CLI Profile on your machine or build host. You can change the default region in your serverless.yml file by setting the region property inside the provider object. You can also deploy to a different AWS Region by passing the --region flag to the sls command. 

You must deploy this solution in the same AWS account and AWS Region where you deployed the Conformity-to-S3 solution.

Exploring Conformity data in New Relic

After the integration is deployed to your AWS account, you should start to see Conformity data in the TMCloudOneConformityEvent custom event reported into your New Relic account. You can explore it with Data Explorer.

Creating Conformity dashboard in New Relic

Let’s build a dashboard to keep tabs on all your AWS account checks reported by Conformity. Complete the following steps:

1. Review the contents of the file: create-dashboard-mutation.txt. Replace all placeholders marked <NEW_RELIC_ACCOUNT_ID> with your New Relic Account ID. This file contains the payload for the NerdGraph mutation that creates the Conformity dashboard in your New Relic account. NerdGraph is New Relic’s unified API in a GraphQL flavor.
2. Go to New Relic's NerdGraph GraphiQL explorer.
3. Create the dashboard by providing the payload for the mutation in the GraphiQL explorer.
4. To open your new dashboard, go to your New Relic account, click Dashboards, and search for it by name (for example TrendMicroCloudOneConformity). 

The following example shows a New Relic dashboard for Trend Micro Cloud One - Conformity (called "Conformity dashboard"), where you can gain insight into Conformity-reported checks and events for all cloud accounts that you have linked.

Tracking New Relic data ingestion

Above the free tier limit of 100 GB/month, you pay for the amount of data ingested into New Relic. See New Relic Pricing for details. To track your data ingestion volume for this integration, you can use the following NRQL queries in the query builder:

Creating a CISO Dashboard

As our final step, let’s create a custom dashboard for chief information security officers (CISOs), accomplished in a matter of a few minutes. The dashboard tracks the top Conformity rules across all your AWS accounts to immediately warn of high severity events such as creating publicly accessible Amazon S3 buckets, Amazon Relational Database Service (Amazon RDS) instances, Amazon EC2 instances, and Amazon Elasticsearch Service (Amazon ES) cluster instances, and activity by the root account or deletion of AWS CloudTrail logs.

As soon as you create the new dashboard, it is immediately available in the smartphone app: