Table of contents
The treasure trove of data we have today can help us uncover trends before everyone else, unlock value where we didn’t think to look before, or blow a hypothesis out of the water, helping us to fail faster and move on to even greater success. And while not every data analysis effort will yield earth-shaking results, it’s a given that data drives far better decisions and helps us understand in greater detail where we need to be focusing our attention.
Now that we all get it, the real question is whether we’re making the most of all the rich information we have or could have at our fingertips. Are you, your colleagues and your company using data to improve your core capabilities, the customer experience, and your own job and team performance? That’s what New Relic wanted to find out in its exclusive “State of the Data Nerd” survey.
The results presented in this report might surprise you. For instance:
- One-third of respondents reported not having the ability to measure and interpret what works and what doesn’t
- Nearly 43 percent of Web-first companies rated themselves ahead of the competition, compared to just 16.9 percent of historically offline companies
- While 13 percent of respondents didn’t release any new code to production in 2014, 41 percent expect to release weekly or faster (daily or hourly) in 2015
Read on to see how companies like yours stack up when it comes to harnessing the power of data. Don’t worry if you feel uncertain because you don’t have enough data (!) about data usage in your company or if you feel that there’s a great deal of room for improvement where you work. You’re not alone. Even Microsoft’s CEO, Satya Nadella, admits that the company is still moving towards a data culture where “every engineer, every day, is looking at the usage data, learning from that usage data, questioning what new things to test out with our products, and being on that improvement cycle, which is the lifeblood of Microsoft.”1
With that, let’s bring on the data!
1“New CEO Nadella Pushes Data Culture at Microsoft”, Bill Riby and Edwin Chan, Reuters, April 15, 2014.
First Things First: About the Survey
In December 2014, using Twitter and email to reach potential respondents, New Relic conducted a survey of people who are responsible for funding, contracting, building, or operating customer-facing software. The intent was to find out how companies are using data to guide investment, product, and customer experience decisions.
Nearly 850 people from more than 725 companies responded to the survey, with a relatively balanced mix of job types, industries, and company sizes:
- Customer-facing versus back-end/internal systems: 64 percent of respondents stated that they are responsible for success in delivering a mobile and/or Web customer experience. The remaining respondents work on back-end or internal systems.
- All sizes of companies: More than half of the respondents (60 percent) work at companies with fewer than 100 employees, while 24 percent work at companies with more than 1,000 employees. Another 16 percent represented the mid-market where company size is between 100 and 1,000 employees.
- A variety of roles: Nearly half of the respondents were developers and software engineers (44 percent), while the second largest group was IT operations with 24 percent. Rounding out the job areas were: marketing, product management, sales, line of business, and security/compliance.
- Web-first versus historically off-line: Web-, mobile-, or softwarenative businesses represented 63 percent of the companies in the survey, while the remaining 37 percent self-identified as historically offline businesses with Web and/or mobile investments.
- Cross-section of industries: Technology was the single largest industry reported, but comprised just 14 percent of the total. Only 30 percent identified themselves as New Relic customers.
In the spirit of full disclosure, respondents were offered a $50 Amazon gift card for completion of the survey, which took an average of 9:02 minutes to complete the 18 questions.
The Journey to Creating a Data Culture
When you think of Coke, you probably don’t think of data, but the beverage giant extracts sales data from its vending machines and correlates it with social media and other cultural information to spot trends in consumption. Michael Conner, senior platform architect at The Coca-Cola Company says, “Without data you’re just a person with an opinion.” 2
At New Relic, we couldn’t agree more with that statement. That’s why we were eager to understand how our customers and other companies with customer-facing software were faring in their efforts to be more data-driven, data-obsessed, and data-savvy. After all, if Coke is using data to improve the vending-machine experience for its customers, just think what companies with online or mobile application software analytics could do.
To find out more, we asked a series of questions about how data was being used to make software decisions. What we learned is that the majority of organizations aren’t yet where they feel they should be when it comes to using data to drive investment or product decisions. Let’s take a closer look.
Takeaway#1: We’re on our way, but still have a long way to go
When asked how their company uses data to make decisions on how to invest, prioritize features, and run customer-facing Web and mobile applications, nearly half of our respondents (49 percent) acknowledged that while they may use data for driving some important decisions, their organizations still have a long way to go to optimize usage of data.
Another 25 percent indicated that their organizations are making good use of data in some places. No data is being used to drive investment or product decisions according to 17 percent of the respondents, with an outsize proportion (55 percent) of these responses coming from traditionally offline companies. Of the 9 percent who believed that their organization was heavily data-driven (kudos!), 75 percent of them were Web-native outfits.
Takeaway #2: Marketing and development are seen as the chief data nerds, but data-driven feedback loops are viewed as successful across groups responsible for software
When asked which department at their company uses data most effectively, the respondents showed a slight self-bias by choosing their own departments about 30 percent of the time. Nonetheless, there’s a perception that marketing (22 percent) and software development (21 percent) top the list of data-savvy departments. The leaders were followed by IT operations (16 percent), sales (13 percent), and finance (4.4 percent). Central IT, HR, manufacturing, and facilities were each seen as leaders by fewer than 5 percent of respondents.
Next we wanted to understand more about the use of data-driven feedback loops to make customer-facing software successful so we asked: “In the process of making customer-facing software successful this past year, which area has the best data-driven feedback loops in your company?” The results indicated that the biggest new opportunity may lie in customer engagement.
In 2014, customer engagement (30 percent), application performance (36 percent), and business success (35 percent) were all seen as relatively evenly successful. But when asked which area would have the most positive impact in helping their organization deliver great customer-facing software in 2015, customer engagement (38 percent) and business success (37 percent) topped the list. For the purposes of this survey, customer engagement was defined as product owners/managers and developers, application performance was defined as developers and IT operations, and business success was defined as customer behavior and marketing or sales.
Takeaway #3: Tech-forward companies feel more confident about data usage
Respondents seemed evenly split between feeling superior or about the same as their competitors (35 percent compared to 33 percent). However, nearly 43 percent of Web, mobile, or tech-native companies rated themselves ahead of the competition, compared to just 16.9 percent of historically offline companies. Remember, of the 9 percent who believed that their organization was heavily data-driven, 75 percent of them were Web-native outfits.
Other findings seem to back up this idea that familiarity breeds confidence when it comes to data usage. A survey by The Economist Intelligent Unit and sponsored by Tableau Software looked at respondent’s use of data compared with competitors and found that 41 percent of respondents believe that in comparison to peer organizations, their organization makes somewhat or substantially better use of data. More than a third of the respondents believed their companies to be on par with data usage compared to their peers or competitors. The top reason cited for why respondents believed their companies were ahead competitors was that “being data-driven is a part of our culture.” 3
The Impact of Data on Time-To-Market
We also wanted to understand the effect data is having on deployment of customer-facing mobile and Web applications. Both the agile development trend and its corollary, DevOps, require rapid, closed loop feedback and for that you need data. With the VersionOne 8th Annual State of Agile Survey reporting that 88 percent of organizations practice agile development,4 we were anxious to see how our respondents were doing in putting the disciplines into practice.
Takeaway #4: Lack of data is a leading inhibitor of experimentation
When asked about what inhibits the ability to experiment with new features and functions for their organizations’ customer-facing Web and mobile apps, 75 percent indicated some type of difficulty that prevents experimentation, while 25 percent said that their organizations are good at enabling experimentation. The number one reason why respondents couldn’t experiment with new features and functions? A limited ability to measure what works and what doesn’t.
Takeaway #5: Agile development is gaining ground, but there’s plenty of room for improvement
Multi-year software projects are rapidly becoming a thing of the past. Today, speed and agility are the watchwords, with teams delivering smaller sets of features more frequently. The goal is to measure projects in terms of days, weeks, and months, with accelerated time-to-value being the ultimate goal.
While studies show the real business benefits of agile development and continuous delivery, the reality is that there’s still quite a bit of room for improvement. For example, 13 percent of respondents in our survey said they didn’t release any code to production in 2014, at least for the apps they are most familiar with. Only 35 percent released code at least weekly, while the rest made do with monthly, quarterly, semi-annual, or annual deployments. In 2014, most companies in our survey released code to production on a monthly basis.
The upside is that many respondents expect to speed their deployment schedules in 2015. Only 8 percent don’t expect to release any code in 2015, while 41 percent expect to deploy weekly or more frequently, up 17 percent compared to 2014.
Interestingly, 58 percent of New Relic customers said they deploy weekly, daily, or hourly. Compare that to 29 percent of non-New Relic customers who said they deploy that often and you can see how access to data provides the insight and confidence that enables rapid iterations and deployment of new features.
Almost All of Us Like Data and Most of Us Like Star Wars
Whether you consider yourself a data nerd, data admirer, or simply a data dabbler, our survey shows that the vast majority of us would prefer to rely on data to improve how we do our jobs. And why not? Data provides us with greater confidence that what we’re doing will have a positive impact on the customer experience, our performance, and the company’s bottom line.
Takeaway#6: A data culture is something that most of us want or expect
We asked survey respondents how they would feel about working at a company where data was used to prioritize work, build and adjust feedback loops, and hold teams and team members accountable. A whopping 87 percent indicated that they’d like or expect to work in a data-driven environment. A small number (11 percent) were unsure, while 2 percent would find it distasteful (really?).
Finally, on a lighter note, we wanted to revisit the age-old nerd question of Star Trek or Star Wars? The majority of us favored Luke Skywalker, Han Solo, and Princess Leia by six percentage points. Not a rout by any means, but a clear advantage for Star Wars. May the force (of data) be with you!
What Do You Think About This Data?
We hope you found our survey results helpful and, at the very least, thought provoking. If you’ve been tooting the data horn for a while in your company and not getting the response you’d like, feel free to bombard your colleagues and management with some of these statistics.
Because life is too short for bad software.
At New Relic, we believe application success is a team sport where everyone needs data to make decisions to create great software. Our suite of software analytics products helps developers, IT operations, and marketing professionals easily make sense of their application performance and customer engagement data.
To learn more about using data to make better software decisions, visit: