Cloud Foundry Diego Approaching 250,000 Containers in a Single Cluster, And What That Means For the Enterprise


By:

September 19, 2016

Interest in containers reached a broad audience nearly two years ago. My, how time flies! Since then, adoption has been, well, up and to the left.

Today, Cloud Foundry is announcing it’s retiring its legacy DEA backend in favor of Diego, which is rapidly approaching the scale of 250,000 containers in a single cluster.

Why should you care?

Consider the iPhone. When you first purchased one, it was likely an 8gb phone. Chances are, you thought you’d never use all the space on that phone. Chances are you were dead wrong. Fast forward to last week’s iPhone 7 announcement. Max size for those devices was 256gb. If you’re anything like me, you probably plan to order the largest capacity device available, just to be safe.

This is what 250,000 containers running concurrently means for the enterprise. If you presume four instances per application, that means 62,000+ applications. Let’s be honest… container adoption just isn’t at this scale in most enterprises today. The data I’ll share below proves just that.

But it will be — very soon. As enterprises continue to move to a digital-first environment, the pace of applications will continue to increase exponentially. And when they do, Cloud Foundry (and Diego) will be ready.

It’s not enough to talk about how many containers you can push into a system. You need to know that the platform will not only build, schedule and run the containers, but care for them over their lifetime. This is the promise of the new scaling metrics for Diego.

And we know the industry is ramping up for a solution just like this. We know because we’re tracking it. About six months ago, Cloud Foundry started tracking container adoption and types of usage. Just last week, we received the latest research.

First, a note on the research itself: it’s statistically significant in every way – spanning eight geographies, covering four main IT and enterprise roles (Developers, Operators, IT Managers and Line of Business) and inclusive of 845 interviews. To put it another way, this is not a survey of early adopters; this is a global survey of the larger IT market.

So, what did we find? In short, container interest in the enterprise remains extremely high, but broad enterprise adoption is slowing as experience with the complexity of containers grows.

Container Adoption

Container deployment chart
Terms defined as follows: Early Deployment = 1) Preliminary evaluation, consideration, or proof of concept (PoC), or 2) trial project/pilot stages; Limited Deployment = 3) Early stage integration for specific business groups or purposes, or 4) total integration for specific business groups or purposes; Broad Deployment = 5) Rolling out across entire company, or 6) total integration across entire company’s processes.

The graphic above shows Container adoption in absolute terms. Early deployment has spiked from 22 to 35 percent while limited deployment has remained steady at 22 percent.

For deployments of all shapes and sizes, the clear preference for container deployment remains platforms. The graphic below highlights the current state of usage for managing and orchestrating containers.

Platforms Remain Strong Preference for Managing and Orchestrating Containers

Cloud Foundry Container Management
Terms defined as follows: Provider-Managed PaaS (e.g., Microsoft Azure, IBM Bluemix, Pivotal Web Services, etc.), Provider-Managed Container Service (e.g., Amazon’s EC2 Container Service, Google’s Container Engine, etc.), Self-Managed PaaS (e.g., Cloud Foundry, Red Hat OpenShift, etc.), Self-Managed On-Premise Orchestration Without Any Third-Party Provider Tools or Platforms (i.e., Do-it-yourself), Self-Managed using Orchestration Tools (e.g., Swarm, Kubernetes, Apache Mesos, etc.).

What’s important for Cloud Foundry adopters to understand is the Diego runtime backend is more than ready for the demands of enterprise production use. For those that have not yet adopted Diego, it’s time to start your transition, especially given today’s announcement from the Cloud Foundry Foundation project teams  about retiring the older DEA backend architecture. Details about the DEA end of life process are available here.

In the next few weeks, we will begin to share the benchmark reports out of the Diego project team. Much like the 256gb iPhone 7, Cloud Foundry is delivering the scale of tomorrow, today.