Exclusive: How Docker Containers Are Being Used Right Now

By now you’re probably familiar with Docker, and how its container technology aims to improve the way you do DevOps—and even how you architect your applications. By encapsulating your compute environments into scripted, lightweight black boxes rather than manually defined and cumbersome-to-maintain virtual machines, Docker has helped light up a whole host of new and faster workflows in Continuous Integration/Continuous Deployment (CI/CD) and microservice communication.

Docker is a technology that New Relic has embraced for years and it now accounts for a majority of our internal production workloads—heck, our lead site reliability engineer even wrote a book on the topic! (See Docker: Up & Running: Shipping Reliable Containers in Production, by Sean Kane and Karl Matthias).

Crunching the Docker numbers

So as we approached the two-year anniversary of New Relic’s original release of support for Docker monitoring, we figured it was time to move beyond the anecdotal evidence to show Docker’s growth, impact, and usage by doing what we do best: crunching some numbers. The following data is based on the hundreds of millions of containers customers have sent through our system and speaks to trends in Docker usage across different industries, company sizes, and use cases.

First, let’s do the fun glamor stats! In the last two years, New Relic has monitored more than 418 million containers (so many that we had to rewrite our analysis tools to accommodate the massive scale of data). In a typical day, we see more than 1.5 million containers, and an average customer has more than 800 containers running at any given moment. Our top ten customers by Docker container usage actually spin up more than 92,000 containers a day.

Docker use cases, from VM replacements to ephemeral computicles

So yeah, Docker seems to have officially transcended the realm of “fun new DevOps toy” to become a full-blown production workhorse. But 92,000 containers a day still sounds crazy, right? Well, as we’ve discussed in the past, some people use Docker as a replacement for their VM infrastructure, but others use it for lightweight, ephemeral compute tasks that spin up, do something, and then disappear (think AWS Lambda). To drive home how prevalent this use case is, we thought we’d share some interesting trends in the lifespan of these containers.

Docker container lifespan by hours, under 24 hours:

The chart above shows that if we analyze only the containers that run for less than 24 hours, the drop-off in usage is stark over time. Literally orders of magnitude more containers run for less than an hour than for several hours (so much so that you can’t even see the long tail on the graph).

Docker container lifespan by minutes, under one hour:

Going even further, the chart above shows just the containers that have run for less than one hour, ranked by number of minutes they ran. We can see that most of these containers last only a few minutes. I’m doubtful that’s a traditional CRM application running in those containers.

Docker container lifespan by hour, from 2 hours to 24 hours:

Above, looking at just the containers that live from 2-24 hours, we can see that things start to get a little more spread out, but the trend toward “shorter is better” remains consistent.

Docker container lifespan by hour, shown on a Log10 scale:

Finally, examining this data via a log scale (since you wouldn’t even be able to see most of the data using a non-log scale) gives a better sense of how container usage is so heavily stacked toward short run and ephemeral.

Docker is changing how code gets written

Whether it’s quick computicle executions like Lambda or the more constant CI/CD of traditional applications, it’s clear that Docker is impacting the way the industry is writing code—for better or for worse. What do you think these trends mean? Tweet at us @newrelic using hashtag #DockerTrends.

Interested in why so many folks choose New Relic to monitor their Dockerized infrastructure? for more details on how we tie containers to what really matters—the applications they support—visit our Docker page or check out New Relic Infrastructure.