Table of contents
The mobile app economy is booming, and opportunity is everywhere. By 2017, Gartner predicts the app economy will reach 268 billion apps downloaded, and $77 billion in revenue.1
But the opportunity to succeed is also the opportunity to fail. There are millions of mobile apps already available for users, with thousands more being launched every month. Users have their pick of what apps to download, and if your app isn’t performing, there’s nothing stopping them from deleting your app and moving on. In fact, it’s been reported that 80 and 90% of mobile apps downloaded are deleted after one use, and 60% of the apps in the Apple App Store have never been downloaded at all.2 These stats are grim even for apps that perform perfectly.
When freezes, crashes, and slow response times occur, however, the results get even more devastating—these are the top three reasons why users write a bad review. Not only can these performance issues cause existing users to delete your app and contribute to bad online reviews, they can prevent new users from downloading your app in the first place.
That’s why having visibility into the performance of mobile apps—in real time and from real users—is so critical; it helps solve customer problems and create great user experiences.
This eBook examines the current state of mobile application performance monitoring; the challenges this is creating for mobile app development; and how the new forms of mobile APM are helping developers create better user experiences for customers and grow their businesses.
1 “Forecast: Mobile App Stores, Worldwide, 2013 Update,” Gartner, September 2013 (https://www.gartner.com/doc/2584918?ref=ddrec)
2 “App Store infested with zombie software, claims analytics startup Adeven,” Gigaom, July 2012 (http://gigaom.com/2012/07/31/app-store-infested-with-zombie-software-claims-analytics-startup-adeven)
The State of the Mobile App Economy
To understand why mobile APM is so important, and so necessary for app developers, let’s first take a deeper look at the drivers and potential roadblocks in the mobile app economy.
The mobile app space is one of the most healthy and growing parts of the tech industry today. Demand for mobile devices is only increasing—from 5.7 devices per household in 2013, to 6+ mobile devices per household in 2014.3 More people than ever before will be able to afford their first mobile device this year. And there are more types of mobile devices than ever to choose from, whether old standards like smartphones and tablets, or new wearable smart watches and glasses.
One of the main reasons for the popularity of all these mobile devices is mobile apps. Demand for mobile apps is booming in every aspect of life, from entertainment to the enterprise.
Currently, around 1.2 billion people around the world use apps. This number is estimated to increase 29.8% each year, reaching 4.4 billion users in 2017.4 The global app economy is estimated to be growing at a rate of 28% between 2012 and 2016. Clearly, the demand for apps is high and is only going to climb higher.
Although the majority of app downloads are free apps, app developers are finding a way to make money through channels such as in-app purchases and mobile advertising, as well as contract development. Contract development may be an especially viable income stream for app developers, because companies in the app economy currently employ an average of 4.5 mobile developers, and this number will likely grow in the years to come.
But despite the opportunities in the app space, there are also some big challenges that lie ahead for mobile developers.
The Fragmentation Problem
One of the biggest problems in the mobile app development space is that you can’t just create one version of your app for all devices. According to a study from Flurry,5 you’ll need to support 331 device models to reach 90% of active users. You’ll need to support 156 device models to reach 80% of users, and 37 device models to reach 60% of users. And this study only covers smartphones and tablets, and current OSes such as iOS, Android, and Windows.
New OSes such as Mozilla’s mobile Firefox OS, a mobile Ubuntu OS, and Sailfish, a proprietary OS created by former Nokia engineers, are already anticipated to launch this year. With new smart wearable tech soon to be available, the fragmentation problem is only going to get exponentially worse.
Currently, most developers are not very aware of or don’t have access to metrics that will show them data sorted by device, OS, and carrier, yet those are exactly the kinds of metrics app developers need in order to pinpoint exactly where problems are arising when apps crash, freeze, or have slow response times.
According to Paul Ryan, CTO at Yellow Pages Group (YPG), some of the greatest mobile challenges his company has faced directly correlated with the fact that they were supporting every major mobile platform. YPG wanted to get in front of as many users as possible, but the company found that troubleshooting across all those different platforms was extremely difficult for them, especially as the live versions behaved in ways they’d never seen during testing.
When you are supporting an app on multiple devices, carriers, and OSes, mobile APM becomes that much more important to the success of your app. But as you will see in the next chapter, many of the methods developers are using to currently monitor mobile app performance are just not enough.
5 “Are Indie App Developers Becoming an Endangered Species?” Flurry Blog, March 2013 (http://blog.flurry.com/bid/94811/Are-Indie-App-Developers-Becoming-an-Endangered-Species)
Why Yesterday’s Mobile Tools Won’t Cut It
While the booming mobile app economy provides a world of new opportunities for app developers (as explored in Chapter 1), it also provides a number of new challenges for developers to deal with, especially in the performance space. In the following pages, we’ll explore the two most popular methods of mobile performance monitoring and why they fall short in today’s complex mobile environments.
Since mobile app development is relatively new, developers are still trying to get a handle on how to best support users. To them, the concept of mobile APM is still quite alien. The most common methods of mobile app performance monitoring today are crash reporting and user testing.
These tools do provide valuable data and should not be discounted. It’s definitely important to understand when and how your apps crash, and how you can resolve these issues. But any normal app that has been built well probably crashes only 1% of the time. What’s happening during the other 99% of the time when the app isn’t crashing? Given that most users will simply delete your app and move on if it fails to perform well, fixing issues after the fact just isn’t enough.
Many times, developers have to wait to hear complaints from users to find out that an issue is occurring—and by that time, multiple users have probably left one-star ratings and bad reviews for the app in the App Stores, which can be greatly damaging to a company’s brand and reputation.
When a company gets a one-star review in one of the app stores, they generally need at least a couple of five-star reviews to counteract it, according to Pierre-Luc Simard, CTO at Mirego, a mobile design and development firm. “Waiting for users to complain can be a very damaging approach to testing a mobile application,” says Simard.
To be able to provide stellar performance, there is a strong need to understand and have visibility into how users are interacting with an app. Being able to dissect the information and figure out which parts are hogging most of the time is key to being able to optimize an app for best performance. With most mobile APM solutions today, there’s a lack of visibility into user interactions. Developers can’t really see what happens on the backend because the necessary metrics aren’t available. And this visibility just happens to be one of the most important aspects of mobile APM that developers have still not fully caught on to.
As mentioned earlier, most well-built apps probably spend only 1% of the time crashing. Being able to get visibility into this data is excellent because it provides developers with valuable insights to help reduce the number of crashes, and provide an optimal end-user experience from that aspect. But what about the other 99% of the time when an app is not crashing?
RunKeeper was using several different tools to monitor the performance of its app: Crashlytics for crash reporting and analytics, Google Analytics for product analytics, Modit for servers and Pingdom for the website. But even with all these tools, RunKeeper still wasn’t getting the visibility into its app performance that it needed to diagnose and fix problems, according to Joe Bondi, Co-founder and CTO of RunKeeper.
It is very important to have deeper visibility into what’s happening when your app is not crashing as well because users don’t want apps that show never-ending spinners, or take too long for screen loads, or throw errors in the backend that the developer has no visibility into. Developers don’t want to be in a ‘you don’t know what you don’t know’ situation.
User testing is another valuable tool developers use to prevent problems with their apps, but the problem with user testing is that it’s almost impossible to test every edge-case and catch all the bugs in your app before launching. Synthetic performance monitoring allows developers to test their apps on multiple devices running different carriers in different parts of the world, but there is nothing like the real deal – real users, scattered in various different regions, running all these use cases that developers probably never thought about. Problems can’t be detected if developers don’t know what to look for in the first place. If you can’t simulate the error, it can be impossible to fix.
At Unwired Revolution, a pilot end user reported an increase in ‘server errors’ when using Unwired’s mobile app, but the company couldn’t track the issue or reproduce it in testing. They ended up having developers comb over log files and debug traces to try and find the problem, wasting many hours of developer time that could have been much better spent.
But even when you are able to detect the bug, it can be just as difficult to then track down the root cause. Unwired Revolution found that when doing user testing for enterprise on-premise solutions, that it was extremely difficult for the company to anticipate and simulate the environmental factors that impact integration. And those environmental factors can lead to unique issues that can take a very long time to debug.
Traditional methods of error diagnosis and performance management simply cannot keep up with the many variables of mobile app performance. There are too many users, too many interactions, too many OSes and device types out there, and not enough time to test them all.
Mobile application testing can be a lot of guesswork, based on the limited evidence at hand. And there’s currently no way to send out a new version of a mobile app to a small subset of users. When new mobile app versions are uploaded to the Google Play or Apple App Store, every single user gets that new version, and can see how buggy it is. The only way you can get feedback on the version is from reviews posted on the app stores. “It’s like driving blind, except you don’t know if you’ve wrecked the car until weeks afterward,” says Mirego CTO Pierre-Luc Simard.
Developers and app providers need to be proactive about detecting problems with their apps, and fixing those problems before users even have a chance to notice. In today’s competitive landscape, developers can’t afford to be left behind.
Why Mobile APM Matters and How to Do It Right
Mobile APM is a crucial part of a business’ app strategy in the competitive app market of today. In the following pages, we’ll explore why good mobile APM is so important, and how to monitor the performance of your app the right way.
Good mobile APM goes beyond just crash reporting, waiting for customer complaints and feedback, andchecking app stores for poor reviews and issues that customers are talking about. You need real-time, real-user information in order to offer real-time solutions to your customers and to create a better user experience.
More often than not mobile apps interface with at least one if not more backend web services to access information. You can have lots of issues at this stage of the process. And with the more common methods of mobile APM like crash reporting, if instances are taking too long to respond on the backend or if a screen is taking too much time to load (and doesn’t result in a crash), there’s no way to find out what actually happened at the code-level. But with newer forms of mobile APM, you can trace the instance all the way from the device through the network and to the backend service, and pinpoint exactly where the problem is.
A good mobile APM solution will, among other things, offer real-time, real-user monitoring of your app, offer a variety of metrics, be able to compare different versions of your apps, and provide the ability to trace an instance from end to end.
Here’s a closer look at some features that a good mobile APM tool should include:
Your app may not be what’s slowing you down—the APIs may be to blame. Look for a mobile APM tool that can monitor all the interactions your app has with external services, and give visibility into how web services and API requests are performing. For API-intensive apps, media or animation-heavy apps, it’s especially important to be able to see response times, data transfers, and throughput.
With your mobile APM solution, you should be able to get visibility into HTTP request errors and network failures, and track error rates by error type and domain. Additionally, you should be able to drill down into details to find the cause of individual errors. This way you can quickly work to resolve the issue and make sure your API or mediaintensive apps—and compliance-sensitive apps, in particular—are working optimally.
Carriers & geography
When you’re managing large-scale distributed apps, you want a mobile APM tool that can break down performance issues by carrier and region. If your app is slowing down only on certain carriers or in certain regions, you need to know, in order to fix the problem. This type of visibility is especially useful for development teams in charge of external and internal enterprise apps.
You need a mobile APM solution that will let you compare different versions of your app, so you can see right in front of you if the changes you made to the newer version of your app are actually creating a better user experience, or if unexpected interactions are causing poor performance. Version analysis is especially important for consumer apps that are custom designed or have media-rich or animation-heavy content, as well as for apps that need frequent updates to refresh dynamic content or to keep up with OS updates.
An efficient mobile APM solution will provide insight into the app’s device-level data, allowing developers to track how their local databases, device memory and the UI are affecting the app’s performance. This helps developers identify the right problems in their gaming or animation-heavy apps, for example, and solve them immediately.
An effective mobile APM solution recognizes that each mobile app has its own unique requirements, and will allow you to custom instrument and monitor any specialized metrics based on functionalities unique to your app.
Potential use cases for custom metrics include wearables/smart object-integrated apps and business apps with specialized transactions. Custom metrics are especially useful for monitoring code that the mobile APM tool can’t normally detect, monitoring code not controlled by your team, monitoring scripts, and monitoring custom events.
To truly pinpoint where performance issues are occurring, knowing how your users are interacting with your app, and how that is affecting performance is a must. With newer mobile APM features like User Interaction Traces you can trace every part of how a user is interacting with your mobile app, from end-to-end. A user interaction trace collects data on memory usage, database metrics, CPU time, and network response times and errors. It also includes method level-traces for each individual task that an interaction is comprised of thereby giving developers access to code-level information. Having access to user interaction traces is especially important if your app is custom-designed for a niche function, if it’s API intensive, or it makes heavy use of media and animation.
Mobile device profiles
Tablets, smartphones, wearable tech—as more and more mobile devices hit the market, it will be even more important for app developers to have a good mobile APM solution that provides device performance metrics, so developers can compare adoption across different devices, see which devices their apps work well on, and which ones have a tendency to be problematic. A good mobile APM solution will also let you track how your apps are performing on different operating systems, and tell you where you can optimize for best performance by monitoring response times, throughput, error rates and user interaction times.
Mobile Use Case: RunKeeper
Mobile fitness platform RunKeeper uses smartphone location technology to help fitness enthusiasts track, measure, and improve their progress toward their fitness goals. The app runs on both iOS and Android and has more than 27 million users worldwide. As RunKeeper has increased in popularity, app performance has become increasingly difficult to predict. RunKeeper used a number of performance monitoring tools, but a significant gap remained in the company’s ability to understand the root cause of emerging issues. It wasn’t until RunKeeper implemented a real-time monitoring solution that it was able to address those issues and improve customer experience.
The Need for Seamless Integration
Mobile APM is a crucial part of your overall app performance monitoring strategy. But it’s only one piece of a larger picture. In the following pages, we’ll explore why it’s important that your mobile APM integrates seamlessly with the rest of your app monitoring environment.
Good mobile APM needs to have seamless integration with the rest of your monitoring environment—web, server, network—as all these pieces complement each other and affect each others’ performance.
Application performance monitoring not only includes the mobile app itself but multiple external pieces, such as web services and servers that interact with and influence your app as well. Different metrics are being monitored in each component, and they talk to each other.
When you have integration between these different areas of performance management and monitoring, it helps zero in on exactly where performance problems are that much more quickly.
A good APM solution will allow you to monitor the performance of each of these pieces and cross-trace across them easily to help identify how they are influencing the performance of one another in order to provide a good end-user experience.
Ideally you’ll want a mobile APM tool that will integrate well enough with your web APM tool to show both on one single-view platform. You’ll be able to see the entire picture of your mobile and web apps working together, and from there, implement end-toend solutions more quickly and more effectively. You’ll also be able follow the greater performance trends of your entire app infrastructure, and work to deliver a better experience to all your customers.
Mobile and Web Use Case: Unwired Revolution
Unwired Revolution is a mobile solutions integrator specializing in security, management, support, connectivity, and application enablement for enterprise mobile initiatives across all major industries. In recent years, the company has turned much of its attention toward the development of custom mobile apps that help clients’ internal teams collaborate more productively and effectively. With a good mobile APM tool integrated with its other monitoring solutions, Unwired Revolution was able to get clear visibility into bottlenecks, helping determine how to allocate subsequent development cycles to improve app performance, and greatly reduced the time needed to diagnose and fix interoperability issues.
Driving Business Growth with Mobile APM
Mobile app performance directly affects the user experience, which means it also directly affects a business’ bottom line through retail performance, ROI, and brand perception. Companies who embrace mobile APM can reap a number of benefits, including faster problem diagnosis and resolution, improved customer experiences, and saved developer time that can then be turned toward innovation.
Web performance has become a key priority in small companies and enterprises alike. As enterprises, in particular, increasingly embrace mobile, mobile APM will become an enterprise priority as well. Here are a few key reasons why it’s a smart move to invest in mobile APM.
Faster problem diagnosis
With mobile APM, companies can identify problems with their apps in real time before a large part of their end-user base even becomes aware of them. Faster diagnosis leads to faster resolution of app performance problems. Developers don’t waste time in going through trial-and-error phases to locate the right problem—the problem is immediately and clearly laid out to them including code-level details. This significantly reduces the mean time to resolution of problems and saves precious developer time.
Better use of internal resources
A company using a good mobile APM tool will eliminate unnecessary support and development team hours spent on diagnosis and repair. The support team will have access to information that helps them diagnose exactly what problems users are experiencing, and the development team will have access to data that helps them solve these problems quickly. Those formerly lost hours can then be used towards innovation in developing new features, improving existing functionality and creating new features that make the app more beneficial to customers.
More satisfied customers
Businesses who proactively monitor their apps can ensure that customers receive a consistently excellent experience every time they use the app. Satisfied users lead to better brand reputation and app store reviews, both of which are crucial in attracting new customers.
Better brand perception
When your customers are happy with their experience, it shows—in better online reviews, greater referral rates, and overall better brand perception as your app earns a reputation for reliable performance.
The above factors—better use of internal resources, more satisfied customers and better brand perception—combine to yield a greater return on investment to any company that invests in a quality mobile APM tool, and implements the results effectively. A good mobile APM tool also helps companies minimize revenue loss by reducing mean time to resolution of problems that could directly result in revenue loss; for example, a purchase transaction that fails to work for no apparent reason.
The True Cost of Customer Dissatisfaction
When mobile users have a poor experience using your app, the consequences can often be devastating. Here’s why:
- 91% of unhappy customers won’t choose to do business with your company again6
- The average cost of attracting new customers is five times as much as it takes to keep current ones7
- On average, one unhappy customer will tell between eight and 16 people about their bad experience, and 20% of unhappy customers will tell more than 20 people8
- With an assumed influence level of 25%, that translates into four lost customers for every customer dissatisfied with their experience
7 Lee Resource Inc
8 Lee Resource Inc.
As the mobile app economy grows, and more and more apps become available to users, app developers will need better methods of mobile APM in order to stay competitive in the marketplace. App developers who rely only on old methods of mobile APM risk losing the good will of current customers and alienating potential new customers with poor performing apps. But with the new mobile APM tools available, there’s no reason for app developers not to have access to real-time monitoring of app performance, giving them the ability to provide a quality app experience to users.
Mobile APM Assessment
Are you concerned about your current method of mobile APM? Take our test below to see if it’s time for a new solution.