KubeCF is a distribution of Cloud Foundry Application Runtime (CFAR) for Kubernetes. The team behind it is extremely happy to report that our project is now in incubation with the Cloud Foundry Foundation. We’re grateful for the support from the Runtime PMC for taking the project under its wing, the members of the CF Kubernetes SIG for providing feedback, and to SUSE for donating this code to the Foundation for open community governance.
The KubeCF Backstory
Incubators are normally for projects in their earliest stages. Even though the KubeCF repository is relatively new, some of the ideas and code have their roots as far back as 2015 when some Helion Stackato engineers figured out how to package Cloud Foundry as containers. Using a bunch of JSON and magic, they were able to get these containers to run with each other on a container platform that used a very early version of Kubernetes under the covers.
The team and code moved to SUSE in 2017, when Helion Cloud Foundry (HCF) became SUSE Cloud Foundry (SCF). The code was refactored to work on Kubernetes directly, and was open sourced under the same Apache 2 license as Cloud Foundry itself. We’ve used this code as the base for our SUSE Cloud Application Platform product, and have customers happily using it in production.
But we’ve also learned a lot in these past few years about packaging CFAR for Kubernetes:
- Tracking upstream releases is difficult when you have to manually patch things to work with Kubernetes
- Kubernetes is missing BOSH features which CFAR currently needs. You can work around these issues on a case-by-case basis, but it’s difficult to maintain.
- Helm is great for deployment, but a Kubernetes operator would really help with platform lifecycle management
With these challenges in mind, and after meeting with SAP and IBM at Cloud Foundry Summit Europe 2017, the first upstream “Containerizing Cloud Foundry” initiative was drafted. This resulted in the creation of Eirini for scheduling user apps on Kubernetes, and Quarks to deploy and manage BOSH releases on Kubernetes. Fissile was also brought into the incubator at the same time (see Chip’s excellent video on how these projects fit together), but one critical component was not included: the actual distribution of CFAR for Kubernetes.
SCF to KubeCF
The central coordination point for upstream CFAR releases is the cloudfoundry/cf-deployment repository. This repository is absolutely essential for bringing all of the components of CFAR together into a distribution.
Until now, the corresponding place for Kubernetes deployments using the Quarks cf-operator was the v3 branch of SUSE/scf. Having Foundation-incubated projects relying entirely on a project outside the CFF is clearly not ideal, so the SUSE team split the v3 branch into its own repo and started prepping the code for donation.
This is the code that has now been brought into the Cloud Foundry incubator under the Runtime PMC.
Kubernetes-native vs. Kubernetes-idiomatic
Cloud Foundry contributors are generally in agreement on how the platform needs to evolve to be relevant in the Kubernetes community, but there are two paths to this destination. One topic that keeps coming up is the transition from “BOSH CFAR” to “Kubernetes CFAR.” How do we build a platform for the future without bringing along baggage we will not need at our destination?
As early movers in this effort, the Quarks and KubeCF teams have worked carefully to stay within the bounds of a BOSH-based ecosystem; not breaking compatibility, not breaking CFF certification, and finding ways to consume all the BOSH releases in use throughout the community. Because of this, the cf-operator became more of a BOSH-operator (or even a BOSH Director for Kubernetes). It provides useful BOSH features which are not available in Kubernetes itself.
This is incredibly powerful, but it’s reasonable to ask if this backward compatibility will be necessary in a future where CFAR component teams are creating releases specifically for Kubernetes.
This is where CF-for-K8s comes in. This repository from the Release Integration team is where CFAR component teams integrate their next-generation releases. There is no BOSH involved, and this forces developers to learn and use Kubernetes tools and techniques. The goal is for the refactored components to become not only Kube-native but “Kube-idiomatic.”
These projects are not in opposition. They are convergent.
Users who want Cloud Foundry on Kubernetes now can get that from KubeCF. It provides the full CFAR experience, passes acceptance tests, and will soon be backing at least two certified commercial releases.
If you’re a developer working on CFAR components, you probably already know about CF-for-K8s. This will be the proving ground for Kubernetes-idiomatic releases as they come out. Over time, component by component, the next-generation versions will be substituted into KubeCF as well.
At some point, the Quarks operator may no longer be necessary to manage the lifecycle of a CFAR deployment. It can be replaced with a more minimalist, special purpose CFAR operator (if that’s even required).
If you’d like to get started with KubeCF, you’re in luck. While I was writing this post the team released version 1.0.0, which is more than ready for you to take for a spin. You’ll need a Kubernetes cluster (v1.14 or greater) with at about 16GB of RAM to spare. If you have some experience with Kubernetes, the Getting Started guide should be enough to get you up and running within an hour or so.
The initial KubeCF team members are all from SUSE, but we want to expand the team to include other contributors from the Cloud Foundry community. If you’re interested in being a part of this, drop us a line in the #kubecf-dev slack channel.
Update: KubeCF 1.0.1
Since this post was drafted, the team has released KubeCF version 1.0.1. It’s a patch version that fixes some of the issues that the community found on version 1.0.0.