Structured logging: What it is and why you need it

Logs are necessary for debugging, troubleshooting, monitoring application performance, auditing security and compliance requirements, and much more. But to get the most out of your logs, they need to be well organized.

Having a consistent, organized log format is known as structured logging. When your logs are organized in a structured format, it’s much easier to search, filter, and analyze the log data to find patterns and fix issues. Yet structured logging can be a challenge, especially when you’re working with complex distributed systems. So, how do you ensure that all your logs are standardized and use the same structured format?

Key takeaways:

• Structured logs are pivotal for debugging, monitoring, and compliance.
• Structured logging enhances efficiency, saves processing power, and supports seamless scaling.
• Structured logs facilitate automation, reduce costs, and are more secure
• New Relic offers powerful tools for log management, adding value to structured logging.

What is structured logging?

Just as it sounds, structured logging is a method of organizing log data in a structured format, making it easier to analyze and interpret. Instead of simply recording raw text, structured logs include key-value pairs that provide context and additional information about the logged event. This allows for more efficient troubleshooting and debugging and better insights into application performance.

Why is structured logging important for application development?

Structured logging is important for application development because it allows for more organized and easily comprehensible logs. Traditional logging methods often result in a jumble of unstructured data, making extracting useful information and troubleshooting issues difficult.

With structured logging, log messages are formatted consistently and standardized, making it easier to search, filter, and analyze them. This not only helps with troubleshooting and debugging but also provides valuable insights into the overall health and performance of the application. It also allows for better collaboration between developers and other stakeholders, as they can easily understand the context and meaning of each log entry.

Benefits of structured logging

Structured logging has numerous benefits. At this point, it’s really more than a best practice. Structure logging is an essential part of the software development process. Let’s take a look at some of the key benefits and why you absolutely need structured logging.

Performance

As your applications scale up, performance becomes even more vital—and let’s face it, even before you scale up, you don’t want your application running slow or taking unnecessary resources. As your application grows, so do the total number of logs. If you're getting logs from many different sources and they aren’t standardized, it’s going to take additional processing power to search and filter them later. Imagine searching millions or even billions of non-standardized log lines with an algorithm designed to catch as many corner cases as possible. You’ll be wasting valuable processing power—not to mention the time of your software teams when they realize they haven’t captured all the potential corner cases.

Improved searchability

How would you like to go to a library with millions of books, but none of them are organized and there’s no way to find the one you want? There’s a reason for the Dewey decimal system, alphabetization, and all the other things you’ll see at a library. Having standardization makes it much easier to find what you’re looking for. The same is true for logs. You might want to search for a specific label (such as for a microservice) or a timestamp (the time a service went down). You might want to do both. It’s much easier to do that if those labels exist and are standardized. If different services have differently named fields—or even worse, no fields for important information at all, it’s much harder to search for what you need.

Consistency

Consistency isn’t just useful for improved searchability. A consistent format also makes it much easier to analyze and compare your logs. Even something as simple as having differently formatted timestamps across services, or differently-named fields, can make it much harder to properly compare how different parts of your application are functioning.

Automation

Structured logs are designed to be much easier for machines to read, which also makes it much easier to automate processes and save time. You’ll be able to more easily integrate your logs with the tools and systems you’re using. Also, efficient automation will cut down costs for your business.

Structured logging best practices

Let’s take a look at some best practices that can help you structure your logs.

• Decide on a consistent log format: Ideally, you should use the same log format across your application. If some of your logs are in JSON and some are in XML or other formats, it’s much harder to maintain consistency across your services and components.
• Use standard data formats: It’s not just about consistency—use a standard data format. Examples include JSON and XML. JSON is great because it’s easy to parse and many tools already use it. By using a widely adopted format, it’s much more likely that the format will integrate well with your services, and there will be plenty of documentation for any issues that arise.
• Include all the context and metadata you need: You’ll want to include basic information such as any IDs, timestamps, and labels that will accurately identify the service, what happened, and when it happened. This makes it much easier to correlate your logs and understand the root cause of any problems.
• Make sure the data is useful for all stakeholders: Especially in a large organization, it’s important to consider a wide range of stakeholders, including developers, DevOps, leaders, SREs, and many other teams. Different stakeholders might have different needs from your log data, so make sure to consult with your stakeholders on what information your logs should contain. Cross-team education on what your structured logs provide is also essential so your stakeholders can actually make the most use of your logs.
• Don’t be too verbose: In general, information is good, but too much verbosity can make it harder to sort, filter, and find the information you need.
• Follow security best practices: Make sure you aren’t logging sensitive data, or if you do need to log sensitive data, ensure that these logs are secure. You can also use log obfuscation to obscure sensitive data.
• Monitor and analyze your structured logs: Your structured logs will only be useful if you’re regularly monitoring them. New Relic is a powerful tool for monitoring, alerting, managing, and making sense of all your logs.

Structured logging example 

We can talk about structured logs and why they’re important all day long, but it would probably be more helpful to see one up close. Here’s an example of a structured log in JSON format:

{  "timestamp": "2024-01-18T10:30:45.123Z",

   "severity": "INFO",

   "message": "User login successful",

   "user_id": 12345,

   "ip_address": "192.168.1.100",

   "application": "MyApp",

   "log_type": "authentication",

   "location": "LoginController",

   "server": "WebServer-01"}

In this example:

• timestamp indicates the time when the log entry was generated.
• severity represents the level of severity (e.g., INFO, ERROR).
• message contains a brief description of the logged event.
• user_id is the unique identifier of the user who performed the action.
• Ip_address stores the user's IP address.
• application specifies the name of the application or service.
• log_type categorizes the log entry (e.g., authentication, error).
• location identifies the code or component where the event occurred.
• server indicates the server or machine where the log was generated.

Structured logs are valuable because they provide a standardized format for log data, making it easier to search, analyze, and extract meaningful insights from the logs. They are commonly used in monitoring and debugging applications and services.

Use cases for structured logs

You’ve learned a bit about the benefits and best practices of structured logging—but what about actual use cases? Let’s take a closer look at some common use cases where you’ll benefit from structured logging. You’ll likely find that many of them are relevant to your application.

• Debugging and troubleshooting: Whether you’re in development, staging, or production, logs are there every step of the way to help you debug and troubleshoot your applications. For example, when an error comes up during development, the stack trace is your friend. Without logs, the stack trace wouldn’t be possible.
• Performance monitoring: Fixing issues is only part of the battle. It’s not enough for your application to work. It should be as fast as possible. Your users will abandon your site within seconds if there are issues with page loads. Structured logs can help you find performance issues and optimize your site.
• Distributed tracing: Distributed tracing is an essential tool for tracking down—or tracing—an issue across multiple different services. You can use structured logging alongside distributed tracing to get a much more comprehensive understanding of how your systems are working.
• Security audits: Security audits are required for many companies to ensure they are compliant—and are also absolutely essential if you have a breach or other security issues. You can use structured logs to better understand user activity in your system for compliance and regulatory purposes, as well as forensic investigations that will help you ensure that previous security issues don’t happen again.
• Business intelligence: What are your users doing on your site? Which pages are most successful, and which need a revamp? You can use structured logging to get valuable insight into user behavior, customer interactions, and other business-related data, which will help your organization make data-driven decisions and improve your products and services.

How New Relic can help with structured logging

New Relic is a powerful all-in-one observability tool that can support all of the use cases discussed in the last section, including troubleshooting issues, performance monitoring, and distributed tracing. So how can New Relic help with structured logs? Let’s take a look:

• Add custom attributes to your logs: With New Relic, you can add custom attributes to your logs when they are ingested, making it much easier to standardize your logs and avoid data gaps.
• Powerful search and analysis capabilities: After your log data is ingested, you can use New Relic Query Language (NRQL) to filter, search, and better understand your log data. NRQL is a query language similar to ANSI SQL. With NRQL, you can fully customize your search to focus on answering the important questions that affect your business goals.
• Centralized logging: An important benefit of structured logging is the ability to compare and analyze all of your logs. But structured logging is only beneficial if all of your logs are in one place. This is a core strength of New Relic: to collect and monitor all of your system data in one place, and to provide the tools to analyze all of it. Learn more about using New Relic for log management. The next image shows the logs UI in New Relic:

• Parsing unstructured logs: While it’s not ideal, you might sometimes have unstructured logs coming from parts of your system. If that’s the case, you can use log parsing in New Relic to pull attributes from your unstructured log data. Then you can use those attributes to more effectively search and query your logs.
• See your logs in context: New Relic provides logs in context, which lets you see your log data in the context of other platform UI experiences like APM, infrastructure monitoring, and distributed tracing. You can use this contextual information to learn more about what’s happening in your application without the need to manually search through your log data.
• Integrations with popular logging frameworks: New Relic allows you to integrate with many popular logging frameworks, including Logstash, Fluent Bit, and Fluentd. That way, you can use the framework of your choice to structure logs and then use New Relic to maximize the benefits of logging.

We hope this guide has helped you understand just how important structured logging is to application monitoring. The insights it provides are invaluable, saving you and your organization time and money by quickly identifying and resolving issues. But managing logs can be a daunting task, especially when dealing with large volumes of data.

That's where New Relic comes in. Our platform offers robust log management capabilities that simplify the process of collecting, analyzing, and visualizing log data from your applications. With our user-friendly interface and powerful features, you can gain valuable understanding into your application's health and make informed decisions to improve its performance.