Authentication log monitoring

Secure authentication allows people to buy products from e-commerce sites, log into social media and email, protect their private information, and much more. Most user-facing web applications include authentication so users can log in securely. If your users have issues logging in to your application, you need to find and fix them as quickly as possible.

What are authentication logs?

Authentication logs are records of events related to user authentication on a computer system, network, or application. These logs typically contain information about when users attempt to log in, whether the login attempt was successful or unsuccessful, and sometimes additional details such as the user's identity, IP address, and the method used for authentication (e.g., username/password, biometric authentication, etc.).

Authentication logs serve several purposes, including:

• Security monitoring: Authentication logs allow system administrators to detect unauthorized access attempts, brute force attacks, and other suspicious activities that may indicate security breaches.
• Forensic analysis: In the event of a security incident, authentication logs can be invaluable for forensic analysis, helping investigators understand how an attacker gained access to a system or network.
• Compliance and auditing: Many regulatory standards and compliance frameworks require organizations to maintain logs of authentication events for auditing purposes. Authentication logs help demonstrate compliance with these requirements.
• Troubleshooting: When users experience difficulties logging in or accessing resources, authentication logs can provide insights into the underlying issues, such as incorrect credentials or authentication system failures.

Why is it important to monitor authentication logs?

There are serious potential issues you can run into regarding authentication. For instance, malicious users may be looking for ways to compromise your application, and security breaches can cost a company millions of dollars.

That’s why it’s important to monitor and log user authentication flows—both to proactively detect issues with authentication when they are happening so you can monitor your server for changes that do not reflect your usage and also to provide a record that you can audit later if your application is compromised.

But how do you log and monitor authentication flows? In this article, you’ll learn:

• Why login monitoring and authentication logs are important
• How to log authentication flows and what information you need to include in your authentication logs
• Which issues authentication logs can help you find and resolve
• How to monitor third-party tools for critical vulnerabilities

The next image shows how an observability platform like New Relic allows you to review your log messages. With New Relic, you can observe patterns in your logs, whether that's issues with billing, recurring errors, or login attempts.

The value of login monitoring using authentication logs

As an observability platform, the New Relic stance on monitoring is simple: you should monitor all the services in your application if possible. That includes logging all the events in your application, including authentication flows. With New Relic, that is as easy as instrumenting your application with an agent and turning on logs in context to automatically start logging all of your application events, including those related to authentication.

Login flows are a very common part of the user experience in many applications. In fact, for applications that rely on user authentication, many of the most important interactions can only happen after a user has logged in. This is true for both user-facing and internal applications. In the case of internal applications, issues with logging in can compromise worker productivity and security.

To make matters more complicated, many companies use multiple authentication systems for different groups of users, such as OAuth and SAML. Third-party authentication services include Okta, AuthO, and Google, and many companies even create custom authentication solutions. Examples include using Rails or .NET libraries for authentication. Combine that with the fact that authentication flows often depend on other pages, navigation bars, and components, which can also potentially lead to issues that make it difficult to log in.

If you’re dealing with multiple authentication flows and systems, problems with any of them can lead to cascading downstream effects. So how do you ensure you’re properly logging and monitoring these flows?

Take these approaches to monitor authentication:

• Set up log management if you haven’t already. If you are new to log management, see How to manage an application log. Log management involves instrumenting your services and setting up log forwarding as needed. Once again, you should be logging all events in your application, regardless of whether they are related to authentication. The exception is sensitive data that you don’t want to log, though you have the option to hide or mask sensitive data.
• Make sure that your authentication logs include attributes that allow you to identify the authentication service where the logs originated. New Relic will handle this for you automatically, but if you are using a custom solution or a platform that doesn’t automatically add identifying data, you should use a standardized format such as JSON and, if needed, customize your authentication logs to ensure they provide useful data you can visualize and analyze. With New Relic, you can also use tags to better organize your data. 
• Logs are more useful in the context of other relevant telemetry data. If you’re using an observability platform like New Relic, use automatic pattern finding to detect unusual patterns in your logs.
• Use a vulnerability management tool like New Relic vulnerability management to detect critical vulnerabilities and exposures (CVEs) as soon as they arise. CVEs in log-in tools can lead to massive breaches of data and unauthorized access to your systems.

What to include in your authentication logs

New Relic observability platform uses agents that will automatically log events for you without the need to install or maintain third-party software. However, if you’re writing your own authentication logs from scratch, they should include the following data:

• Who: If a user logs into your application, the log should identify that user.
• What: What action the user took, such as a successful or failed login attempt.
• When: All logs should include a timestamp, including authentication logs.
• How: Which login service was used? This information is particularly important if your application uses multiple login services.
• Where: The log should include the IP address of the user. This can help you pinpoint issues that are specific to certain regions.

You’ll also want your log data to be structured as opposed to unstructured. You can parse structured logs, making it easier to filter and search your logs.

Here’s a basic example of an unstructured log:

6/15/2022 15:34:03: Log in user ‘Albert Einstein’ success

While this includes a timestamp and basic login information about a user, it doesn’t include important information such as the IP address or the service being used. Also, it doesn’t use a format like JSON to make the data easily parseable. That would make it tough to actually analyze authentication log flows if a problem comes up.

Here’s an example of a structured log that’s easier to parse and includes more data:

6/15/2022 15:34:03: { actionType: ‘logIn’, service: ‘rails bcrypt’, user: ‘Albert Einstein’, ip: 1.1.1.1 result: ‘success’ }

As you can see, the data is now organized into key-value pairs like actionType and service. This makes it easier to filter and search your log data by specific keys. You could filter by specific services to see how each service is performing, filter by actionType to specifically look at the latency or error rate of ‘logIn’ actions, and so on.

What if you don’t want to go to the trouble of formatting, creating, and outputting custom logs? With New Relic, you can automatically instrument all the services in your environment and set up logs in context.

Get started with Log Management using New Relic.

What you can potentially find using standardized authentication log reports

Monitoring your authentication flows can help you detect these issues:

• Security threats: For instance, you might see a spike in login attempts or failed login attempts, or in the case of a DoS attack, you might see fewer login attempts.
• Login errors: A spike in errors or an increase in failed login attempts might indicate that there’s an error in the login process. By ensuring there are logs at each step of the process, including visits to the sign-in page and redirects to success or error pages, you can pinpoint exactly where issues are happening in the process.
• Login trends: Analyze trends in your login data, ranging from page latency to total sign-ins over time.
• Manage data for compliance: For data compliance purposes, you may be required to store log data such as security data for an extended period of time. Often security incidents don’t come to light until months or even years later, and you can review archived log data as needed.

With New Relic, you can automatically start monitoring your entire application in minutes, including your login flows, and then visualize and analyze your data using prebuilt dashboards. 

Best practices for monitoring authentication logs

Monitoring authentication logs is critical for maintaining the security of your systems. Here are some best practices for effective monitoring:

1. Centralized logging: Aggregate authentication logs from all relevant sources (e.g., operating systems, applications) to a centralized logging system. This allows for easier analysis and correlation of events.
2. Regular review: Regularly review authentication logs for unusual or suspicious activity. Set up alerts for specific patterns or events that may indicate a security breach.
3. User behavior analytics: Use user behavior analytics (UBA) tools to detect anomalous behavior in authentication logs, such as unusual login times, locations, or access patterns.
4. Multi-factor authentication (MFA): Implement MFA to add an extra layer of security. Monitor MFA logs for any unauthorized attempts or bypass attempts.
5. Log retention and archiving: Retain authentication logs for a sufficient period (based on regulatory requirements or internal policies) and ensure they are securely archived to prevent tampering.
6. Access controls: Limit access to authentication logs to authorized personnel only. Use role-based access control (RBAC) to enforce this.
7. Regular updates and patching: Keep your authentication systems and related software updated with the latest security patches to protect against vulnerabilities.
8. Integration with SIEM: Integrate authentication logs with a Security Information and Event Management (SIEM) system for advanced correlation and analysis of security events.
9. Regular audits: Conduct regular audits of authentication logs to ensure compliance with security policies and standards.
10. Incident response plan: Have an incident response plan that includes procedures for responding to security incidents identified through authentication logs.

Monitoring third-party authentication tools

While some teams create their own authentication flows from scratch, it’s more common to use third-party tools for secure log-ins. However, while third-party tools can be convenient and safer than custom solutions (it’s often best to leave security and authentication to companies that are experts in those areas), there is a significant downside. 

You simply don’t have the same visibility into third-party tools, and even trusted companies deal with security breaches. How can you handle critical vulnerabilities in third-party tools when you lack visibility into how they work (if they are closed source), their maintenance schedules, and more?

While logging authentication flows can help you pinpoint issues that arise in your third-party tools, you also need to continuously evaluate the security risk of all the external libraries that your systems use, including authentication tools. With New Relic vulnerability management, your third-party libraries are continuously evaluated for security risk. You can also use built-in quickstarts to integrate security tools like Snyk, Lacework, GitHub Dependabot, AWS Security Hub, and Aquasec Trivy with New Relic.

Get started with the New Relic vulnerability management docs.