Why Your Cloud Spend Is Too Damn High

When I talk to New Relic customers, they typically ask two related questions:  "How can we improve our customer end-user experience?" and, "How can we reduce our overall operating costs in the public cloud?"

The first question focuses on stability, performance, and customer adoption. The second question comes from customers with sticker shock after they’ve been hit with a massive bill from their cloud service provider.

Both questions apply to customers in a wide variety of industries, from the restaurant and hospitality industries to government agencies to the financial sector. Each sector has its own particular issues: In the financial sector, for example, customer experience is critical not only from a profitability perspective but also from a regulatory perspective. If these organizations don't provide a good customer experience, then in addition to losing customers, they risk enormous fines from the regulators. Government agencies, meanwhile, have to provide their citizen-users with excellent services in a cost-effective manner, or they might end up on the evening news.

Scaling for the two-minute promise

To see how this works, let’s imagine a restaurant business fixated on the “two-minute issue.” The idea is that if a take-out or delivery customer can't place an order online or via mobile app within two minutes, the restaurant risks losing that customer—and the associated revenue—to a competitor. The restaurant chain uses multiple cloud providers in multiple regions and is busy adopting modern cloud-native practices, like containers and microservices— rearchitecting on the fly. Just as important, the restaurant chain also has to keep its franchisees happy, which means controlling costs for thousands of small businesses on notoriously tight margins.

To manage these costs, the business runs scalability testing every morning against its production environment for the expected workloads that day. But unexpected things are always expected in the food-service business. If traffic is bad or commuter train service is interrupted, for example, a lot of people get home late from work and decide to order dinner instead of cooking. That can lead to an unplanned spike in order volume—requiring the business to scale out its cloud services to make sure customers can still place their orders in that two-minute timeframe.

At this point, you might ask: "What's the problem, isn’t the public cloud infinitely scalable?” Well … sure. But technical complexities can make it difficult for public cloud vendors to respond quickly and efficiently to unexpected spikes in traffic. For example, modern web apps often include dozens of different microservices in the technology stack, integrated with a bunch of external services, like mapping services, GPS services, payment gateways, etc., which can’t all be spun up in time to meet that two-minute deadline.

To avoid this scenario, the restaurant chain scales for peak load, based on historical averages for the current and prior week, and then adds a 20% buffer.

Now, it’s over-provisioned.

Over-provisioning in the cloud costs money—a lot of money

So, in the quest to satisfy customers, the organization has over-provisioned its cloud usage, which eats into those razor-thin restaurant margins.

Companies try to minimize over-provisioning in a number of ways. Autoscaling technologies, PaaS providers, and container orchestration platforms like Kubernetes can help companies reduce over provisioning, but they can’t completely solve the problem.

When you're doing hundreds of thousands of transactions a day, each triggering a separate function, that cost can become significant. But in the hyper-competitive online world, many companies feel they have no choice but to over provision to avoid impacting the all-important digital customer experience.

How New Relic can help

Here's the bottom line: Reducing the need for over-provisioning can help cut operating costs in the public cloud, but it can’t come at the expense of the customer experience. Fortunately, New Relic can help in a variety of ways.

New Relic enables our customers to understand how their users experience websites and applications. This insight helps organizations quickly identify if a user’s experience has been degraded or the service is unavailable. New Relic also provides insights into the resource usage for services, enabling our customers to perform performance-based cost optimization for all their cloud services—without impacting the customer experience.

New Relic does this by connecting and analyzing the relationships between complex services—focusing on answering, "How can we improve our customer end-user experience?" and, "How can we reduce our overall operating costs in the public cloud?" Recent product enhancements such as the New Relic Kubernetes integration and New Relic Lambda integration, for example, make it easier than ever to understand performance and resource utilization in a single view.

Once you have the insights and relationships correlated between application performance and resource utilization, you can right-size your services and optimize your cloud costs.

Finally, New Relic is continually innovating to provide our customers with detailed information on the relationship between your customers’ journeys and the cost of providing those services.