Navigating tool fragmentation to consolidate your observability tools

In the fast-paced world of technology, observability into the software applications that run the products and services that businesses deliver to their customers is critical for business success. But as many organizations pursue improved monitoring and observability solutions, they encounter the challenge of fragmentation across data, tools, and teams. According to the 2022 Observability Forecast, these challenges aren’t just problems that engineering teams need to solve—these are business problems.

Observability is a way to check how well a system is working and spot any problems or issues by looking at the information it shares as external outputs. This information, also known as telemetry data, includes metrics, events, logs, and traces (also known as MELT). By using this data effectively, you can make informed, data-driven decisions and take the right actions to fix any issues. It’s ideal to find mistakes before going to production, so observability is for both the developer and operations side of DevOps teams.

And observability reaches beyond DevOps. Whether you’re an engineer developing a system, you’re an SRE ensuring it’s running optimally, or you’re on the business side providing teams with resources to build products that meet customer needs, you face the challenge of managing a complicated mix of data and tools required to keep your systems secure and running smoothly. 

According to the 2022 global survey of 1,614 technology professionals, the state of observability is often characterized by using multiple tools, incomplete coverage of the entire tech stack, and various complex data streams and systems to coordinate. There’s also a pattern of relying on manual effort and the reactive approach of incident tickets to detect problems. Respondents revealed room for improvement in outage frequency and the time to detect and resolve issues.

This blog post gives a tour through the fragmented landscape of observability tools. The intended destination is a unified approach to observability. You'll learn examples of unified telemetry, visualizations, and dashboards. You’ll learn about how organizations gain knowledge about system interruptions, and some of the challenges they face. But you’ll also get example use cases and benefits. 

The fragmented state of monitoring tools

Starting out in the current state of observability, organizations often rely on multiple tools to monitor the health of their systems, leading to fragmentation and increased complexity.

• 82% used four or more tools, and 94% used two or more.
• Only 2% relied on a single tool for their observability needs.

This multi-tool landscape highlights the inherent challenges in achieving full-stack observability, such as siloed data and switching between interfaces. In fact, 25% of survey respondents noted that too many monitoring tools are a primary challenge that prevents them from prioritizing/achieving full-stack observability.

Telemetry data: Silos vs a unified view

The survey explored different approaches to managing and unifying telemetry data in organizations, revealing that: 

• Half (50%) said their telemetry data is at least somewhat siloed (they silo telemetry data in discrete data stores), including 8% who said entirely siloed.
• Just 7% said their telemetry data is entirely unified (they store it in one place).

In addition, only 33% said their telemetry (metrics, events, logs, traces) is unified in a single pane for consumption across teams.

Siloed and fragmented data can lead to a frustrating experience for teams who can't get access to use that data to make decisions and resolve issues. You’ll also likely face increased costs, a lack of context, and slower troubleshooting.

Visualizations and dashboards: Fragmentation remains

When it comes to visualizing telemetry data, organizations are still grappling with how to unify visualization and dashboards.

• About a quarter (23%) said they had more disparate visualization and dashboarding solutions, without cross-communication.
• A mere 13% of respondents said they had entirely unified visualizations and dashboards, which means that all their telemetry data is visualized in one dashboard solution.
• More than half (53%) had not deployed custom dashboards at all.

If you have multiple monitoring tools and dashboards, identifying the root cause of an issue invariably requires swivel-chair integration, where you throw different tools onto multiple screens and swing your chair from side to side to get a full view of what’s happening in your environment. 

Detecting software and system interruptions

According to the report, detecting software and system interruptions is also challenging. Almost half (46%) of the survey respondents primarily learned about these interruptions through multiple monitoring tools.

Even worse, the process is largely manual for many organizations. A third of respondents said they still learn about interruptions and outages mainly through manual checks, tests, or complaints.

Given the relative frequency of outages (72% experience low-business-impact outages and 52% experience high-business-impact outages once per week or more), the findings of how often manual effort and incident tickets are the sources of knowledge for these outages are noteworthy.

Preferences: Single platform vs multiple point solutions

While the preference for a single observability platform is evident, implementation often falls short.

• Almost half (47%) of the respondents favored one consolidated platform for observability instead of multiple point solutions, which implies that many organizations want an all-in-one approach.
• Despite this preference, 94% still used multiple monitoring tools, and a mere 2% used a single observability tool.

In recent years, observability vendors have developed specialized tools to help engineering teams monitor specific parts of the tech stack. This has led to increased complexity due to different experiences and data storage methods across tools.

A unified platform for all types and sources of telemetry, like New Relic, is the best way to harness the full potential of observability. Then engineering teams can view and collaborate on all entities and dependencies in one place, eliminating silos between teams, tools, and data.

The current state of observability is characterized by multiple tools and fragmentation. But there's a growing strategic preference for a single, consolidated platform to overcome the obstacles posed by tool fragmentation in achieving full-stack observability.

Benefits and use cases for observability

The observability forecast highlighted several potential benefits of and use cases for observability. Two were specifically applicable to the challenges of tool fragmentation:

• Observability helps with consolidating IT tools.
• Observability aligns with cost-cutting efforts.

Benefits of observability

More than a quarter (28%) of respondents mentioned that observability helps with the consolidation of IT tooling. This result highlights the significant impact observability can have on transforming an organization’s business, technology, and revenue.

Other benefits of observability include improved uptime, performance, and reliability, as well as better operational efficiency and customer experience. 

Use cases for observability

A similar percentage of respondents (27%) noted that observability is crucial for supporting cost-cutting efforts through tool consolidation.

Other use cases for observability include optimizing cloud resource usage and spending, enhancing digital customer experience to gain competitive advantage, and managing containerized environments.

This result highlights the diverse range of use cases where observability can significantly benefit your organization.

The roadblocks to full-stack observability

In the observability survey, respondents highlighted several challenges related to tool fragmentation that prevented them from prioritizing and achieving full-stack observability:

• 25% reported too many monitoring tools as their primary challenge.
• 24% blamed a disparate tech stack.
• 21% mentioned siloed data.

Overall, the survey respondents expressed a strong preference for a single, integrated tool that would simplify their workflow and enable them to focus on more valuable tasks. To overcome the side effects of fragmentation—such as poor customer experiences, rising IT costs, inefficient resource allocation, and security risks—consider consolidating your systems, tools, and information sources into a unified observability platform. Ensuring consistent, secure, and available digital experiences for your customers is crucial for success.

How can New Relic help you with observability tool consolidation?

The 2022 Observability Forecast uncovered an ongoing struggle around tool fragmentation and a desire for a more consolidated solution. If you’re looking to benefit from observability, you must address the challenges posed by multiple monitoring tools, siloed data, and disparate visualization and dashboarding processes. Embracing a single, unified platform can help both engineering teams and businesses reduce costs, consolidate IT tooling, foster more efficient operations, and make better decisions.

The New Relic all-in-one observability platform can help you collect data including crucial information about system performance, transactions, and customer experience—displayed in dashboards that all your various teams can use. Then you can implement alerts to notify the right teams and improve decision-making. With New Relic, you can keep watch over your applications (known as application performance monitoring or APM) as well as your infrastructure, and you can manage your logs in context.

Learn how others have consolidated tools with New Relic.