Control data privacy with New Relic log obfuscation options

New Relic log management log obfuscation options help you ensure data privacy and make it easy to follow your organization's log security guidelines.

Log obfuscation fundamentals

Data obfuscation is a methodology used to hide all or parts of a data record to protect sensitive log data such as personally identifiable information (PII), access tokens, or any other private or regulated data. 

Our log management service automatically masks patterns for credit cards and Social Security numbers. With these new obfuscation options, you can use regular expressions and create custom rules to hash or mask confidential data before the information is stored. Using regular expressions (regex) means you won’t need lengthy manual configurations.

Better logs, easy security, and compliance

Our logs in context functionality extends visibility by showing logs alongside metrics to troubleshoot issues faster while including any additional filtering to follow your organization's security guidelines to mask, obfuscate, or prevent sending any sensitive data all without the need for a lengthy manual process or custom configurations from your teams.

What data should be obfuscated in logs?

Here are a few examples of private data you might want to obscure:

• Personally identifiable information (PII):  information like Social Security numbers, combinations of data, like first name and date of birth or last name and zip code, or other user-generated data that is considered confidential.
• Protected health information (PHI): Health data, such as medical records.
• Financial data, like credit card numbers.
• Passwords.
• IP addresses may be considered sensitive, especially when in combination with PII.

Note that this is not an exhaustive list. Be sure to follow your organization's security guidelines to see what log data you may be required to protect.

Log obfuscation benefits

Log obfuscation can provide significant benefits for organizations that handle sensitive data, including: 

• Data protection: Obfuscation reduces the risk of data breaches and enhances data privacy and security.
• Compliance: Log obfuscation helps organizations comply with data protection regulations that require protecting personally identifiable information (PII) and other sensitive data. 
• Customer trust: Log obfuscation enhances customer trust by demonstrating the organization's commitment to data privacy and security. 
• Data governance: Log obfuscation improves data governance by ensuring that sensitive data is properly managed and protected throughout its lifecycle. 
• Data analysis: Log obfuscation enables data analysis by allowing organizations to use sensitive data in log files for analysis and troubleshooting without compromising data privacy and security.

Get started with obfuscation rules

Masking vs hashing

To prevent sending PII, PHI, or any other data that needs to be secured, you can choose one of two methods:

• Masking is one-way, permanent obfuscation of the data. The data will be obscured and replaced with x’s (such as XXXX, instead of your data). Once this is done, there is no way to undo it or recover the original string.
• Hashing is two-way obfuscation, where the data is hidden by using a Secure Hash Algorithm 512 (SHA-256) string. A hashing tool in the UI allows customers to look up their SHA-256 by entering the original text. The user can then search for that SHA-256 string in the logs UI.

Create an obfuscation expression

Define regular expressions to specify which data to hide. Use the following options to create an obfuscation expression:

1. Go to one.newrelic.com > Logs and from the left navigation, select Obfuscation.
2. Select Create regex.

Enter a name for your new obfuscation rule and a regular expression matching the sensitive data you want to capture. Use RE2 syntax.

Create an obfuscation rule

Hide sensitive data using matching criteria:

1. Go to one.newrelic.com > Logs and from the left navigation, select Obfuscation.
2. Select Create obfuscation rule.
3. Enter a name for your new obfuscation rule and matching criteria (in NRQL format) to capture the target set of logs you want to obfuscate.
4. Add new actions (the first one is added automatically) to specify the obfuscation expression (regex) to capture each set of attributes and whether to mask or hash them. Multiple attributes can be specified comma-separated. Mask will replace all matching characters with the letter x. If you use mask, you won't be able to query for a particular obfuscated value later. Hash will replace sensitive data with the SHA-256 hash value. If you use hash, you will be able to query them using our hashing tool, provided you know its unhashed value.

5.  Select Create rule to create and activate your obfuscation rule.

   You’ve now successfully created a rule to mask sensitive information before data is stored in NRDB.

Log obfuscation best practices

Keep your organization effective, efficient, and compliant with these best practices: 

1. Identify sensitive data: Identify the types of sensitive data that need to be obfuscated in log files. 
2. Develop an obfuscation strategy: A clear and consistent strategy defines how each type of sensitive data will be obfuscated in log files. 
3. Implement automated obfuscation: Implement an automated process for obfuscating sensitive data in log files. This makes the process more consistent and reduces the risk of human error. It also allows for more efficient and scalable obfuscation of large volumes of log data.
4. Test and validate the obfuscation: Test and validate the obfuscation process to ensure that sensitive data is properly obfuscated and that the log files are still useful for analysis and troubleshooting. 
5. Monitor and audit the obfuscation process: Monitor and audit the obfuscation process to ensure that there are no unintended consequences or gaps in the obfuscation process. 
6. Document the obfuscation process: Document the steps in the process, including the obfuscation strategy, the tools and techniques used for obfuscation, and any relevant policies and procedures.

Get started with log obfuscation