Table of contents
Is there any doubt that the name of the business game today is software? As Vijay Gurbaxani, University of California professor and founding director of the Center for Digital Transformation, writes in the Harvard Business Review, the current business environment requires every business to view itself as a software business. And, as such, every business must keep its eye on the proverbial software ball.
It seems obvious that the quality and performance of your company’s software must be spot on or your customers will look elsewhere. Yet, take a look at these statistics:
- 75% of organizations indicated that application performance is an issue for them, with 25% claiming their application is outright slow.
- 50% of development and IT professionals reported that their organizations have repeatedly experienced slowdowns and outages due to traffic spikes.
- 40% of consumers indicate that they will leave a page that takes longer than 3 seconds to load, but the average U.S. retail mobile site had a load time averaging 6.9 seconds in July 2016.
What is going on with modern software development that’s causing performance and quality to be major pain points for many organizations? The easier question might be “what isn’t going on?” From far higher expectations for constant innovation to increasing complexity of the application environment, software teams today—developers, operations engineers, product managers, and others—face greater challenges than ever as they race to deliver innovative, quality software quickly.
It’s no wonder then that application performance monitoring (APM) has become an essential component of building and running great software. And it should come as no surprise that in addition to all the other skills and capabilities that the software team must perfect, using APM to its greatest effectiveness should be high on the list.
This ebook introduces you to APM, what an APM journey looks like, and how you can apply APM to improve the quality and performance of your software, and, in doing so, drive business outcomes that deliver a positive impact to the bottom line and help your company stay ahead of the competition.
Chapter 1: The Pressure’s on for Today’s Digital Businesses
The fundamentals of how we develop, test, deliver, and operate software have shifted dramatically in recent years and continue to evolve and morph at an impressive speed. Agile development, cloud computing, DevOps, software analytics, microservices, containers—the list of changes and trends just keeps growing. The upside is that these evolving technologies and approaches are all contributing to software teams’—and by extension, their companies’—ability to move faster.
Here are a few of the big changes impacting software development and operations teams today:
Culture change: The DevOps approach. The DevOps movement continues to grow, with 74% of companies adopting a DevOps culture. By moving to DevOps, companies are aiming to better align software and IT teams with business goals and value. DevOps helps bridge the gap between development and operations, fostering communication and collaboration for faster delivery of higher quality software.
As teams move to a DevOps approach, they are also gaining greater autonomy and accountability. With that accountability comes adherence to performance metrics. You won’t know if your DevOps approach is successful unless you measure and monitor deployment speed, application performance, customer experience, and business success.
Forecast: Everything’s cloudy. When it comes to speed, many organizations rely on cloud computing to streamline and accelerate the development and deployment of software. The downside (yes, there is one small disadvantage) is that you lose some control and visibility compared to on-premises infrastructure. And hybrid clouds make application visibility even more challenging as applications and transactions suddenly span across completely different infrastructures, some of which is beyond your four walls. To make up for this drawback, you need an APM solution that gives you end-to-end visibility no matter where parts of your application reside.
Break it up: Microservices. With a microservices model, large applications are broken into small, loosely coupled, autonomous services. Small changes can be made quickly to code with reduced risk, which improves agility, speed, and efficiency of development while enhancing the resiliency of applications. The downside (yes, once again, there is a disadvantage) is that while microservices help reduce the complexity of larger, monolithic applications, a microservices-based architecture also introduces the challenge of tracking, visualizing, and monitoring the relationships between multiple services. You need visibility into dependencies and overall architectures to gain understanding and clarity in a microservices environment.
Think inside the box: Containers. A 2016 survey showed that the percentage of companies running containers more than doubled in one year’s time, from 38% to 79%. The same survey reported the biggest drivers of container adoption to be increased developer efficiency and support for microservices. That’s because containers provide a dynamic infrastructure that lets developers deliver new functionality in smaller units, accelerating the time from idea (or fix) to consumption by users. Because different pieces of an application can be swapped out or scaled up without affecting the rest of the system, software teams can roll out releases without downtime.
A perfect storm of complexity and pressures
In addition to the trends above, new platforms and frameworks, infrastructure solutions, and services hit the market all the time, which means the technology stack is constantly evolving. Plus, the software environment is increasingly diverse, with a mix of languages, datastores, cloud services, third-party services, mobile technologies, and other components. At the risk of stating the obvious: the modern software environment is more complex than ever (see Figure 1).
While all the trends and changes we’ve identified are beneficial in the main, the challenges for software teams are non-trivial: deliver high-quality software rapidly to meet increasingly greater demands and higher expectations from the business and customers. For IT operations, it’s increasingly difficult to monitor production environments because teams lack end-to-end visibility into all the components that make up complex application stacks. Without deep insight into application performance, developers spend more and more time troubleshooting and fighting fires instead of being able to focus on creating and improving the digital customer experience.
As the software environment and development processes continue to change, teams that don’t have the data they need will fall behind competitors that do have the visibility and insight to:
- Tame the complexity and gain a deeper understanding of application and user behavior
- Accelerate troubleshooting and resolve issues faster
- Make more informed decisions about the impact of changes on the customer experience
- Use data to share perspectives in order to improve collaboration between IT operations and development teams
Chapter 2: Modern Software Needs Modern APM
Given the trends and challenges that software teams face today, it’s unimaginable that some of them are still working in the dark, with little visibility into the performance of their applications and, more important, how application performance affects users.
To deliver truly great software and exceptional digital experiences, you need to get to the truth about your applications’ performance, your customers’ experience, and your software’s impact on the business. With APM, you can “turn the lights on” and uncover the hidden issues you need to work on and discover opportunities for optimizing your applications.
When you have real-time insight and analysis of historical trends, suddenly you can ask all the right questions:
- How can I optimize and improve the performance of my software?
- Were top accounts affected by today’s slowdown?
- How do response time trends impact customer usage?
- Are power users engaging with our new feature?
- Are my users having a good experience with my software?
- How many customers upgraded last week?
- Can my application architecture and infrastructure scale if traffic suddenly spikes? (A Kardashian endorsed your product, OMG!)
A modern APM solution lets you overcome the challenges of delivering better software faster in a more complex environment. You get visibility that doesn’t stop at the edge of one technology, but spans your entire application stack. And APM provides a common language that fosters understanding and collaboration across the software team as well as the rest of the business.
Top APM Benefits
- Greater understanding of the entire application ecosystem helps you more effectively manage and optimize your software.
- Real-time, specific data about performance issues helps you quickly understand the scope of incidents and resolve them faster for less downtime, reduced business impact, and happier customers.
- Knowing exactly what went wrong using root-cause analysis helps you prevent the problem from recurring in the future.
- Pinpointing which parts of your application to proactively optimize lets you improve the customer experience and create a positive outcome for the business.
- Shorter feedback loops means your software teams can pivot and iterate quickly to innovate and deliver great customer experiences.
Chapter 3: The APM Journey
While the current environment necessitates the need for a robust APM solution, to get the greatest value from APM, we believe it needs to be viewed more as a journey than as a tool. Which means, of course, that everyone has to start somewhere on that journey, and then progress to reach the desired end state.
What does that desired end state look like? It’s having the data and the knowledge you need to use your software to transform your business. Our APM journey starts with visibility (shining a light into the dark corners of your applications), adds troubleshooting finesse, moves up to optimization, and progresses to full insight and proactive planning.
In the first phase of APM adoption, your organization is learning what’s going on in your application environment. You’re gaining the visibility you need to keep your applications humming along and your users engaged. That visibility is actually twofold: it’s broad and horizontal, looking across your architecture, application environment, application dependencies, external APIs, and more; it’s also deep and vertical, giving you insight into your application code, errors, application stack, database calls, and the correlation between your application and the underlying infrastructure.
In this phase, you get visibility into:
- Dependencies between applications, infrastructure, and third-party services so you can understand how your app and the infrastructure work together, how your app interacts with the application stack/architecture, and how your technology stack interacts with other parts of the infrastructure.
- Application performance and availability so you know whether your app is up and working, how fast or slow it is, and what type of experience end users are having.
- Usage, including infrastructure utilization, apps, users, and resource consumption as well as throughput.
- Changes, including deployments, infrastructure upgrades, configuration, and traffic and usage patterns. These give you insight into the impact of changes, the scope of the change, and performance deltas before and after the change.
- Transactions, where you can see what the app is doing compared to what you were expecting it to do. With the right visibility, you can see end-to-end transactions and performance of those transactions.
In this phase, your organization learns how to use APM to improve troubleshooting, making it far faster and more efficient to resolve issues more quickly for users while helping software teams spend less time fighting fires in reactive mode and more time creating new software and optimizing the digital experience.
You’ll use APM to optimize your firefighting tactics, giving your teams the data they need, when they need it, and in the form they need. This allows them to quickly determine the scope and cause of production issues and help them validate fixes.
For this phase (see Figure 3), you’ll focus on improving the entire troubleshooting process:
- Initial awareness: Use APM alerts and proactive monitoring to get timely awareness of potential issues.
- Scope of impact: Determine the scope of impact based on timing and severity.
- Diagnosis: Focus your investigation using APM data to pinpoint the source of the problem quickly and precisely.
- Remediation: Attempt to fix the issue.
- Validation and communication: Create a feedback loop process and confirm the fix.
Now that you’ve conquered the troubleshooting monster and firefighting is quicker and easier, you have more time for proactively improving performance and availability. In this phase, your organization will start using real-time and historical data you get from your APM solution to optimize performance and business results by making your applications and services:
- Faster: Using your APM data, identify and prioritize areas of the application for optimization. These incremental performance improvements can go a long way towards delivering a better customer experience.
- More stable: Using your APM data, tackle areas of the application to improve stability. This will help proactively reduce the number of fires you need to fight in the production environment, freeing up more time for … further optimization and innovation.
Now your organization is optimizing for change, allowing teams to experiment and innovate more often and at a faster pace. Core optimization efforts take place across the software development lifecycle:
- Development: Optimization during the development phase includes: advanced planning for architectural considerations, security, load testing, and initial infrastructure configuration.
- Release: Optimizations during this phase are focused on production readiness, including setting up alerts, understanding the impact on production upon release by using deployment markers, reviewing performance data in staging/pre-production, feature flags, and more.
- Production: In this phase, optimization efforts include scaling, proactively identifying errors and monitoring user behavior, monitoring performance of third-party API calls, and understanding key transactions.
In this last phase of APM maturity, your organization is learning how to use the gamut of data from your APM solution to scale and evolve your applications and infrastructure to transform your business. You’ll do this by:
- Re-architecting your apps and environment to take advantage of the latest technology advances such as cloud computing, microservices, and containers.
- Supporting organizational changes such as DevOps and agile methodologies.
- Planning for growth and scalability to deal with seasonal/opportunistic traffic spikes.
- Leverage data to gather insights from application and user behavior to augment and drive the product roadmap.
- Reporting on the impact of your software on business results.
APM assessment questions
- Are we measuring actual end-user experience?
- Are we meeting our SLAs?
- Is our app delivering consistent response times across users and geographies?
- Is our app delivering a consistent user experience across browsers and mobile?
- Can we quantify how third-party technology and services impact our app’s performance?
- Can we monitor and troubleshoot problems for our mobile app using the same APM tool we use for our web applications?
Chapter 4: What’s in It for Your Business?
You know the technical benefits of APM—improved troubleshooting, faster resolution and reduced MTTR, fewer performance and stability issues, and improved productivity for your teams, to name a few—but what’s in it for your business?
Let’s look at some real-world examples to give you some insight into what you can expect when you embark on the APM journey.
Rocketing to the top of the industry
With more than 50 million people visiting its websites monthly, 10 million followers on social networks, and operations in Mexico, Argentina, and Brazil, Netshoes Group is the largest pure-play e-commerce sports retailer in the world. Key to Netshoes Group’s astonishing growth has been its consistently innovative use of technology. From the 3D scanner system that allows customers to compare the sizes of different shoe models (know your size in every brand!) to a mobile app that identifies tennis shoe models by photos and then enables customers to purchase them via their devices, Netshoes Group is constantly looking for ways to improve the customer experience through technology.
“IT is the body and soul of Netshoes Group. Because we are a 100% e-commerce company, technology is our lifeblood—and it’s not just about doing flashy things on the website. It’s also about making sure that site navigation and usability are constantly optimized, so that there are no bottlenecks and that the website performs exactly as expected. Only then can we rest assured that we’re providing the best possible purchase experience for our customers,” says Eduardo Berti, infrastructure director at Netshoes.
“Every minute that our site is slow or we’re unable to deliver value to the consumer, we lose revenue and fail to achieve our primary goal—the sale of products. New Relic [APM] helps our team save an enormous amount of time and energy diagnosing problems and monitoring our technology,” says Berti.
Dramatically improving the customer experience
Advanced Learning, a division of Advanced Business Solutions, specializes in developing software for the educational market. In 2012, Advanced Learning introduced Progresso, an application designed to streamline and optimize almost every aspect of running an educational establishment, from scheduling classes to tracking payroll. “Parents are more likely to access the application on evenings and weekends, whereas staff members use it both during the day and into the evening,” says Martin Reynolds, application architect and development manager at Advanced Learning. “It’s pretty much a 24/7 deal.”
When the company realized its new application wasn’t delivering the stellar performance expected of it, Reynolds’ team created a plan. “With help from New Relic [APM], in that first sprint we achieved a 25% improvement in the performance of our application, a 30% improvement in server performance, and a 25% reduction in error rates. The changes have been so dramatic that our customers have noticed, and they’ve been very appreciative,” says Reynolds.
Delivering innovation faster
With more than 360 businesses, the Hearst Corporation is one of the largest media and communications conglomerates in the United States. Leveraging its 128-year history as one of the world’s most admired private media and information companies, Hearst has embraced the digital revolution. The IT teams throughout Hearst Corporation have adopted DevOps, agile development, and microservices to support the company’s fast-paced digital innovation.
By incorporating New Relic APM throughout the development and deployment process, Hearst has improved deployment frequency as well as quality. “New Relic helps us get features out faster to our customers,” says Dave Swift, vice president of engineering at Hearst Newspapers. “And we reduced our rollbacks from many to almost none.”
Allen Duan, operations manager in the office of the chief technology officer at Hearst, believes that New Relic improves the company’s ability to experiment with new technology to deliver new customer experiences quickly. “New Relic is a key partner in enabling us to try new things,” says Duan. “As we experiment on new platforms, whether it’s Snapchat, Discover, or some other new technology, we can instrument and build the right experience that lets us reach our audiences as quickly as possible.”
Chapter 5: Choose the Right Tool for the Job
Achieving the technology and business benefits we’ve talked about as your company progresses on its APM journey requires the right vehicle: a robust, modern APM solution. You need one that gives you real-time visibility into the entire stack of your digital business, including applications, infrastructure, and the overall architecture.
Despite claims to the contrary, not all solutions are up to the challenge. Here’s what you need in an APM solution to achieve the results we talked about in Chapter 4.
With a software-as-a-service (SaaS) solution, you can change your architecture, give access to APM data to anyone who needs it, add new applications, scale up, or scale down without worrying about updating your monitoring infrastructure to keep up. Look for a pure SaaS APM solution for easy and fast deployment, lower total cost of ownership, and myriad other benefits (see Figure 4) that on-premises APM software doesn’t deliver:
Having multiple monitoring solutions for different languages and environments is a headache and limits your ability to trace transactions across applications, databases, and servers. Look for an APM solution that offers support for all the languages and environments you use and are considering using in the future so that you can avoid having your monitoring data stuck in silos of different monitoring tools.
End-to-end visibility across the entire stack
From the browser to the backend. From mobile apps to infrastructure. You need insights—broad and horizontal as well as deep and vertical—for every part of your technology business. Insist on an APM solution that lets you follow the performance of a critical transaction across your entire service-oriented application environment. This way you can visualize where your app is spending its time and identify even the tiniest transaction details.
Look for an APM solution that gives you a code-level visibility of application performance, which provides insights on where applications are spending the most time and CPU usage, which helps you optimize performance and resolve problems fast. You should be able to drill down for instant insights into specific code segments and SQL statements.
Dependency mapping and visualization
Picture this: one centralized map that lets you visualize performance metrics for distributed applications, architectures, and external services. Look for an APM solution that lets you easily understand how your distributed apps and services are performing and communicating, including upstream and downstream dependencies and connections to data stores and external services.
Securing a monitoring system probably isn’t your company’s core competency. But it should be for cloud-based monitoring companies. Look for an APM provider with world-class security teams, and make sure they have the investments, commitment, and audit results to prove it. You should also choose a solution that allows your organization to have complete visibility and control over what, if any, sensitive information is processed by the service. Other musts for security include:
- Independently audited
- AICPA SOC2 Tier III certified
- Cloud Security Alliance – Security, Trust & Assurance Registry (CSA STAR)
Chapter 6: Getting the Most From APM
Taking advantage of proven best practices can help your organization move along the APM journey faster, with fewer frustrations and wrong turns.
1. Get to know your Apdex scores
Apdex is an industry standard to measure users' satisfaction with the response time of web applications and services. It gives application owners better insight into how satisfied users are, in contrast to traditional metrics such as average response time, which can be skewed by a few very long responses. Set up alerts based on your Apdex score, so you know immediately when there’s a user satisfaction problem.
2. Track deployment history
When development teams are pushing new code out the door as frequently as possible, it can be hard to measure the impact each deployment is having on performance. One way to stay in tune with how all these changes are affecting your application is to use deployment reports within your APM solution. Deployment reports should list recent deployments and their impact on end users and app servers’ Apdex scores, along with response times, throughput, and errors. You should also be able to view and drill down into the details and catch errors related to recent deployments, or file a ticket and share details with your team.
3. Identify and set up key transactions
Depending on the nature of your application, some transactions may be more important for you to measure and monitor than others. Your APM solution should allow you to closely monitor what you consider to be your app’s most business-critical transactions. For key transactions, you’ll want to track metrics such as response time, call counts, and error rates. Consider setting alert thresholds so that you’re notified when a key transaction is performing poorly.
4. Enable single sign on
To simplify password management for your employees and strengthen security, you may already be using SSO with your other systems. You should do the same with your APM solution. Choose an APM solution that offers SSO integration, enabling account administrators to enforce strong passwords and restrict login via a corporate authentication mechanism.
5. Add labels to your applications
When you’ve got several different applications using the same APM account and each application spans multiple environments (e.g., development, test, pre-production, production), it can be hard to find a specific application in your overview dashboard. Adding labels to your apps lets you organize apps and servers by segmenting them into logical groups. The most common labels used by mature APM users are application name and environment. For example, if you wanted to view the billing application in test, you could simply filter by “billing app” (name label) and “test” (environment label).
6. Ask your own questions
Beyond relying just on the standard dashboards and reporting in an APM solution, you should consider creating your own dashboards where you can ask new questions and query the data in different ways to help you better understand application performance, customer experience, and business success and how they all interrelate.
In today’s digital world, software is one of your biggest differentiators. For your company to rule its corner of the world, you need to be delivering amazing software, with unbeatable performance and availability.
Combining the right APM solution with both the learnings of the APM journey and APM best practices, your software teams can deliver products that overtake the competition, keep customers coming back for more, and make the business appreciate the value and positive bottom-line impact of your innovative software.
Check out these additional APM resources
Troubleshooting App Slowness with APM
Troubleshooting Errors with New Relic APM