CNCF Annual Survey Report Review: The state of cloud and Kubernetes

If there was one thing that struck me from the Cloud Native Computing Foundation (CNCF) Annual Survey Report, it's simply how far Kubernetes has come in such a short time. And the delightfully alliterative subtitle says it all: "2021: The year Kubernetes crossed the chasm."

The excitement is understandable. Whether you read the headline as hyperbole or not, the report—with real-world quantitative data provided by New Relic and Datadog—isn't wrong. Kubernetes is that rarest of technology initiatives—something that started off as an interesting idea, morphed into a technological ideal, was implemented in a closed environment, and finally was presented to the world as an open-source goodwill offering. Although its original capabilities left something to be desired, its potential captured people's imaginations. This project has come into its own as the CNCF has continued to listen to the community, making improvements both large and small.

The result is a real solution to real problems, executed in such a way that Kubernetes is not only viable but reasonable enough to reach a state of ubiquity.

What does this year’s report tell us? 

The data on usage and adoption show prevalent uses for Kubernetes:  

• For production workloads (not just sandboxes or tests)
• In public cloud spaces (not just private cloud or the laptop-under-the-desk)
• By “regular” companies (not just Software as a Service vendors) 
• With observability increasingly becoming a requirement for all of the above. 

But that’s just the high-level read. Look closely at the report, and you’ll see a nuanced story start to emerge.

Developers and organizations alike are maturing in their approach to containerization

The rise of Kubernetes—a 67% increase in developers using K8s with a 49% increase in overall container adoption since last year—is directly connected to the increasingly mature way organizations are using containers. Additionally, it indicates a change in the way organizations do development overall. The rise of non-SaaS vendor companies learning what containers are and how they should be used means most organizations have finally recognized what a horrible idea “lift and shift” ever was. They’ve stopped trying to P-to-V (physical-to-virtual) “MyBigCorporateApp” into a container and then shove it into the cloud, only to lose their collective minds the following month when their AWS bill has So. Many. Zeros. Leaders in both dev and ops teams have successfully made the case to management and proven the value of comprehensively refactoring so apps aren’t just “container friendly” but also leverage the benefits of the new structure and context.

With this improved understanding of what containerization is and how it’s best used, comes the inevitable desire for consistent and repeatable orchestration, reliable scalability, and widespread observability across the entire stack. This is, of course, exactly where Kubernetes starts to shine.

Organizations aren’t interested in rolling their own

However, the report uncovers another, more subtle, key point. While individuals, teams, and entire organizations have a growing appreciation for what Kubernetes does, they aren’t interested in rolling their own. 

I liken this to storage management—nobody ever wants to manage a storage array. (Well, except loveable weirdos like Stephen Foskett. But we like him anyway.) As soon as “just click this button and we’ll do the rest” storage options came on the scene, everyone jumped at them. For Kubernetes, as much as folks like Keith Townsend, Kelsey Hightower, and others have tried to break down the process of rolling your own, the overwhelming response was, “Could we just, you know...NOT, instead?”

Here at New Relic, we’ve seen a parallel and related rise in providers who offer Kubernetes-as-a-service and the voracious appetite of consumers using these services. In fact, 92% of New Relic customer accounts that are using Kubernetes are managed. This data suggests that many organizations are using and may even prefer using managed service providers to handle the underlying infrastructure.  

But wait for it, there’s a punchline: Our research further shows that this could lead to users not understanding what they are using or knowing how Kubernetes works under the hood!

Do you know what this sounds like? Electricity. Plumbing. Basically any commoditized service. Companies used to have dedicated staff for auto mechanics, copier repair, telecom specialists, (mutters under his breath “desktop IT support”), and the like. But eventually, it became unnecessary as the service itself became more widely available, which caused installations to become more consistent, all of which made the service simpler to use.

Kubernetes may well be going the same way, to the point where few people will need to know how it works. While this race-to-the-bottom commoditization would be a death knell in a for-profit context, it’s absolutely a win when it comes to open-source solutions like this. 

Anyone who’s worked in IT for more than 15 minutes knows how quickly predictions can go off the rails, but I’ll take the risk: I believe the time has arrived for Kubernetes to not only shine but reign supreme in its space. New Relic’s “O11y Trends Report for 2022” underscores this point: 

However, I’d be remiss if I didn’t highlight the particular challenges in that type of growth. It comes down to the hard truth that you can’t know if something is “better” if you have no idea how it was running before or what “normal” looks like. And the only way to get that is through observability (o11y). Once again, from the O11y Trends report:

Per the CNCF Annual Survey Report, the rise of MSP (managed service provider)-like options for Kubernetes is pushing businesses even further “up the stack” in terms of development techniques and processes. Businesses are starting to behave more like cloud-native organizations, even if that isn’t their core business. They’re adopting more continuous integration and continuous delivery (CI/CD) behaviors, increasing the number of production updates in a cycle and abandoning traditional monitoring tools and techniques in favor of robust observability solutions. 

There are more insights to be found in the CNCF Annual Survey Report. But the main takeaway is one that warms the heart of this grizzled (and sometimes cynical) IT veteran: individuals, teams, and organizations are maturing in their use of containers and cloud technologies. Folks are (finally) putting into daily practice the hard-won lessons from DevOps colleagues. Best of all, these practices mean organizations both reap the benefits of these habits and improve the overall landscape of cloud computing.