Using modern observability to establish a data-driven culture

Most organizations create and operate digital products using multiple clouds and complex software architectures. These investments in architecture empower customer engagement models and customer experiences, and CEOs now realize that their technology architecture has become their business architecture. Great customer experiences are aligned with revenue growth, renewal rates, customer satisfaction, and high net promoter scores (NPS).Now more than ever, customers demand high-quality digital experiences that can be accessed anywhere, anytime.

To achieve these demanding business objectives and overcome the complexity challenges that multi-cloud architectures and cloud-native applications present, IT leaders (including developers, DevOps teams, platform engineering, security, operations, and SREs) are deploying observability solutions. These solutions provide unified data collection, analysis, and visualization across mountains of data (metrics, logs, traces, and events pools) to identify, resolve, and prevent downtime and outages before they impact customers. Observability is fast becoming a modern model for maintaining prominent levels of system reliability by using data and analytics to dig into performance problems, with data access across different IT teams that must make fast decisions with accurate, trusted data from a sole source.

The value (and access) of the performance data has become increasingly important to drive decision making, creating a foundation for a data-driven culture. Some large IT organizations IDC advises have started to use observability as a foundation for their SRE, DevOps, development, and CloudOps data-driven decision making. They’re taking advantage of the observability data to collaborate and make faster decisions that help improve system reliability and customer outcomes. Most IT executives want to move their culture from reacting to problems to a more proactive posture, getting in front of pending operational problems before customer impact.

Key practices to establish a data-driven culture 

Key practices using observability to establish a data-driven culture include: 

• Collecting and analyzing the right data in a unified, trusted fashion that is accessible across IT and business teams and stakeholders.
• Having transparency of the customer (and employee) journey and related friction points as they interact with the business and products.
• Being able to define who the customers are, what they value, and opportunities to collect customer feedback and share back to developers and business stakeholders so they can improve products quickly.   
• Reduce the time to answer performance-related problems from days or hours to within minutes.
• Empower and scale SRE teams using observability data to create proper service-level indicators (SLIs) and service-level objectives (SLOs) parameters that drive best-fit budget discussions with lines of business. 
• Empowering cross-team collaboration and trust in the underlying numbers supporting SLIs, while expanding the partnership with the business. 
• The ability to keep honing SLOs to match system reliability requirements, using data to drive continuous improvement. 

Here’s an example that we’ve seen. A large financial services firm wanted to ensure that its multi-cloud and cloud-native environments provided customers with a high-performing experience. The business had several fragmented tools and data lakes, and limited problem-identification processes, so teams were limited in their ability to see end-to-end across their critical products and services. They didn’t have enough data to provide a clear picture of the customer experience. By investing in observability and focusing on a data-driven foundation, they quickly created detailed metrics on customers’ experiences. They increased collaboration across development, operations, and platform engineering teams, using data to drive performance decisions, and improving overall system reliability. 

In addition, processes became well defined, and roles and responsibilities across teams became clearer. The leadership team strategically chose to focus on observability as a method to improve team collaboration, use data and analysis in a more impactful manner, and improve the customer experience and NPS scores. They recognized the need to move from monitoring to observability across people, process, and technologies. In this case, observability and data-driven decisions drove considerable technology and business improvements, with more teams working on observability than initially anticipated. Now the business is working with IT teams to define business metrics that best fit the technology capabilities observability offers. 

Observability is no longer optional

IDC believes observability is a foundational capability for a data-driven culture across development, operations, SecOps, SRE, and platform engineering teams. In a world where digital products and experience dictate success or failure, observability is no longer optional. Smart executives will drive business advantages with observability and a data-driven culture.