By the time you are reading this, the cf-operator might have been renamed to quarks-operator to match the team name. However this article will keep referring to it as ‘cf-operator’.
With KubeCF, a full Cloud Foundry on Kubernetes, releasing 1.0 and 1.0.1 and joining the Foundation Incubator, let’s take a closer look at the cf-operator from the Quarks team.
In short, the cf-operator can deploy BOSH releases on Kubernetes. The cf-operator follows the principles set out for Kubernetes operators. It’s a collection of controllers to manage custom resources and react to changes in a cluster.
How?
BOSH releases are converted to Docker images and the deployment manifest is embedded in a Kubernetes custom resource. The operator watches for this ‘boshdeployment’ resource in a namespace. Once it is detected, the operator interpolates BOSH variables, renders BOSH jobs ERB templates and finally converts the outcome to Kubernetes core resources, like statefulsets, services and jobs.
The operator also supports updating a deployment and modifies the affected Kubernetes resources accordingly.
It is KubeCF’s purpose to provide the cf-deployment manifests wrapped into custom resources. Helm 3 is used for templating and parametrization of those manifests. It first installs the cf-operator, then it creates the ‘boshdeployment’ custom resource, ops files and variables.
Not a Monolith
The name might be misleading, but the cf-operator doesn’t have any special knowledge about the Cloud Foundry Application Runtime (CFAR). While it extends Kubernetes with support for BOSH manifests, we didn’t want to name it ‘bosh-operator’ as it also offers more generic features, like secret generation, secret rotation, improved jobs and statefulset management.
Looking at the operator’s features in detail, compatibility with BOSH releases and deployment manifests is the most important part. However, the design is much more modular than previous solutions, i.e. SCF, and the intention is to use the operator to maintain a consistent approach to deployment of Cloud Foundry Application Runtime, while the Cloud Foundry components themselves become more Kubernetes native.
Amongst those modular parts are QuarksJob, QuarksSecret and QuarksStatefulset.
QuarksJob
QuarksJob is already a standalone component, which can be installed via its own helm chart. It introduces a qjob custom resource definition, which wraps a Kubernetes (K8s) job. QuarksJobs can act as a template and create K8s jobs when they’re triggered. They can also store the job’s container output in secrets. The generated secrets can be ‘versioned’, which means that every time the job runs, instead of updating the existing secret, a new one is created.
QuarksSecret
QuarksSecret cannot yet be run without starting the full cf-operator. Its qsec custom resource definition describes how the K8s secret should be generated, as a password, a certificate (which is optionally signed by the clusters CA), or as an SSH key.
QuarksStatefulset
Finally, QuarksStatefulset wraps a K8s statefulset in the qsts custom resource and adds canary, zero-downtime deployments, zones and active-passive probe support.
All the Kubernetes Resources You Need
Interoperability doesn’t stop with the standalone components. By relying on core Kubernetes resources and concepts, the whole system is very open and extensible:
* Implicit BOSH variables are read from K8s secrets.
* Explicit variables, e.g. password, certificate, and SSH keys are created as K8s secrets.
* BOSH links can be fulfilled by existing K8s secrets.
* BOSH link properties are created as K8s secrets, and any consuming pods are optionally restarted if properties change.
* Endpoints and services are added for instance groups.
* Kubernetes zones are used for BOSH AZs.
* Secret rotation for individual k8s secrets can be done in-cluster.
I hope this article gave some insight into the structure and capabilities of the cf-operator controllers. If you have any questions, you can find me on #quarks-dev Slack.
