Introducing EiriniX: How to Build Eirini Extensions

By: | July 19, 2019
Share

Introducing EiriniX

Working with Eirini, Cloud Foundry and Kubernetes is really exciting.

The possibility of directly leveraging Kubernetes features with the Cloud Foundry ecosystem opens up new scenarios and completely new ecosystems around Eirini.

At SUSE we’ve built a framework called “EiriniX” to provide a common way to build Extensions around Eirini, using the Kubernetes API.

When you want to provide new features around Eirini, it means interacting with the Kubernetes API. This can be complex and repetitive work that takes precious time. EiriniX allows you to focus on the logic of your extension and get your features working in a real Kubernetes cluster.

EiriniX is currently used in an extension that provides persistence support (Persi) to Cloud Foundry Eirini apps by using Kubernetes Persistent Volume Claims.

In this article, you’ll learn how to build and test a simple Extension for Eirini with EiriniX.

Note: The full sample described in this article can be found in the eirinix-sample Github repository [1].

Mutating webhooks

The Cloud Foundry Quarks Project [2] inspired us to use webhooks as a mechanism for extending Eirini in a Kubernetes “native” way.

EiriniX sets up a Kubernetes Mutating webhook [3][4] for each Extension, so each can also be seen as a standalone Kubernetes operator of sorts, targeting Eirini applications.

We can look at mutating webhooks as a way to intercept Kubernetes API requests. When the Kube API server receives a request that matches one of the rules provided by the registered mutating webhooks, the API server sends an admission review request to the webhook and the webhook replies with a response patch. EiriniX provides you with a framework for handling the setup that’s specific to Eirini applications.

The core of an Eirini Extension is to provide a set of patches against an Eirini Application Pod that is about to run in the Kubernetes cluster.

Create your first Extension

Enough jibber jabber! Let’s create our first Extension and then run it on Minikube.

Let’s say we want to inject a new environment variable to every application that’s pushed to Cloud Foundry. Let’s call this environment variable EXAMPLE.

NOTE: The same approach can be used to do much fancier things, like injecting a sidecar container into the pod.

Requirements

  • GO >=1.12
  • Minikube [12]

Let’s create a directory in the system for our extension, hereinafter called eirinix-helloworld, and we assume the repository import path is github.com/eirinix/eirinix-helloworld. It will be composed of two files: main.go which contains the main() for the binary, and hello/helloworld.go, where we’ll be implementing the Extension logic.

Here’s our directory structure:

eirinix-helloworld
├── hello
│   └── helloworld.go
└── main.go

1 directory, 2 files

1) Create hello/helloworld.go

We will sketch up the logic of our Extension first, so let’s create hello/helloworld.go and populate it with the following content:

package hello

import (
     "context"
     "errors"
     "net/http"

     eirinix "github.com/SUSE/eirinix"
     corev1 "k8s.io/api/core/v1"
     "sigs.k8s.io/controller-runtime/pkg/webhook/admission"
     "sigs.k8s.io/controller-runtime/pkg/webhook/admission/types"
)

type Extension struct{}

func (ext *Extension) Handle(ctx context.Context, eiriniManager eirinix.Manager, pod *corev1.Pod, req types.Request) types.Response {

     if pod == nil {
          return admission.ErrorResponse(http.StatusBadRequest, errors.New("no pod could be decoded from the request"))
     }
     podCopy := pod.DeepCopy()
     for i := range podCopy.Spec.Containers {
           c := &podCopy.Spec.Containers[i]
           c.Env = append(c.Env, corev1.EnvVar{Name: "EXAMPLE", Value: "EiriniX is awesome!"})
      }
      return admission.PatchResponse(pod, podCopy)
}

An EiriniX Extension is a GO structure that defines a Handle function as declared in the sample code. This returns a types.Response structure, which is the patch response that Kubernetes understands. That’s the only requested method by the EiriniX Extension interface [5].

In our case, the response is derived from the difference between two pod structures: the original pod that we receive from the request, and the “desired” pod that contains our changes. We create a deep copy of the pod in another variable using pod.DeepCopy(), and we make all changes on this copy. Later on we compute the patchset from the two structures by calling admission.PatchResponse(pod, podCopy).

2) Create main.go

Now we have to create a main.go that starts our Extension. We just need to register our Extension and run the EiriniX Extension Manager.

package main

import (
     "os"
     
     eirinix "github.com/SUSE/eirinix"
     helloworld "github.com/eirinix/eirinix-helloworld/hello"
)

func main() {
     x := eirinix.NewManager(
          eirinix.ManagerOptions{
          Namespace:           "default",
          Host: os.Getenv("EIRINI_HELLOWORLD_EXTENSION_SERVICE_SERVICE_HOST"),
          Port:                4545,
          OperatorFingerprint: "eirini-x-helloworld",
     })
     x.AddExtension(&helloworld.Extension{})
     x.Start()
}

Here we create a new EiriniX Manager:

  • namespace is set to “default”, meaning that we’ll look for Eirini app pods in the “default” namespace only
  • With
    code>os.Getenv("EIRINI_HELLOWORLD_EXTENSION_SERVICE_SERVICE_HOST")

    we consume the ip from the associated service

  • OperatorFingerprint is a unique identifier for your operator.

 

Note: Among the EiriniX Manager options, you can set up a default Failure policy (see [6][11]). If set to fail, whatever error could occur in your Extension would cause admission to fail, and no pod would be started. By default, the EiriniX policy is to Fail.

We then add our Extension to the manager, which runs an http server for handling mutation requests. You can add as many extensions as you like using the same process.

x.Start() starts the main loop. It returns an error in case there are any runtime issues (failure to connect to k8s, etc. )

3) Build it!

$> echo 'module github.com/eirinix/eirinix-helloworld' > go.mod
$> go get github.com/SUSE/eirinix
$> go build

You should now have a new binary in your project folder: eirinix-helloworld.

Note: if you run into dependency problems with go mod, just copy the whole file in the sample repository [1]

4) Test your Extension in a Kubernetes cluster

EiriniX allows you to run your component inside a Kubernetes cluster without caring about connection details.

We will now run the Extension that we’ve built inside a pod in a Kubernetes cluster. The pod requires special permissions to be able to contact the Kubernetes API server and to modify pod definitions. Refer to [9] for the complete example.

Let’s build a Docker image that contains our extension, so we can run it inside a Kubernetes cluster.

Create a Dockerfile with the following content:

FROM golang
WORKDIR /tmp/build
ADD . /tmp/build
RUN cd /tmp/build && \
     go build


ENTRYPOINT ["/tmp/build/eirinix-helloworld"]

Then build the image:

$> docker build --rm -t eirinix-sample-extension .

Note: you need to make sure this image is available on the Kubernetes cluster you’re running on. If using minikube, you can use

eval $(minikube docker-env)

to build the image directly on the node.

 Set up permissions and run the extension:

We are ready at this point to run our extension in the kube cluster:

$> kubectl apply -f https://raw.githubusercontent.com/SUSE/eirinix-sample/master/contrib/eirinix-sample.yaml

Verify that the extension is running:

$> kubectl logs eirini-helloworld-extension

2019-06-10T13:12:46.130Z        INFO    eirinix-sample/main.go:27     Starting 0.0.1 with namespace default

2019-06-10T13:12:46.141Z        INFO    config/getter.go:56     Using in-cluster kube config

2019-06-10T13:12:46.142Z        INFO    config/checker.go:36    Checking kube config

2019-06-10T13:12:46.194Z        INFO    ctxlog/context.go:51    Creating webhook server certificate

2019-06-10T13:12:46.194Z        DEBUG   in_memory_generator/certificate.go:21   Generating certificate webhook-server-ca

2019-06-10T13:12:53.833Z        DEBUG   in_memory_generator/certificate.go:21   Generating certificate webhook-server-cert

Check that the mutating webhook is in place:

$> kubectl get mutatingwebhookconfiguration

NAME                                        CREATED AT

eirini-x-helloworld-mutating-hook-default   2019-06-10T13:12:58Z

5) (Mock) Test it

Let’s try to create a pod that looks like an Eirini app. We will spawn a busybox image that sleeps, so we can inspect it.

$> cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
     name: eirini-fake-app
     labels:
          source_type: APP
spec:
     containers:
          - image: busybox:1.28.4
          command:
              - sleep
              - "3600"
           name: eirini-fake-app
     restartPolicy: Always
EOF

After the pod is running, inspect the pod – you should be able to see our environment variable set by the Extension:

$> kubectl describe pod eirini-fake-app

...
Status:             Running
IP:                 172.17.0.4
Containers:
eirini-fake-app:
Container ID:  docker://244dc2a2c7a36d078549b0d43899fff60bdb3cd53bf13927d3a6024b42a0ac01
Image:         busybox:1.28.4
Image ID:      docker-pullable://busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
Port:          <none>
Host Port:     <none>
Command:
sleep
3600
State:          Running
Started:      Thu, 13 Jun 2019 08:49:10 +0200
Ready:          True
Restart Count:  0
Environment:
EXAMPLE:  Eirinix is awesome!
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-m97vc (ro)
...

 

Great! The hard part is over – all you have to do now is come up with new feature ideas.

Note: In a real use case (at SUSE) we developed persistence support and we consumed the extension in the eirini-bosh-release [7][8].

A “pluggable” Eirini Ecosystem

With Extensions defined as separate logical pieces, we gain the ability to plug features that we need into our deployment. With an extensible approach, features can be easily reused and shared within the community without having an impact on the development of Eirini core features.

This post made possible with the help of my colleague Vlad Iovanov.

References

[12] Install MiniKube – https://kubernetes.io/docs/tasks/tools/install-minikube/

Ettore Di Giacinto Profile Image

Ettore Di Giacinto, AUTHOR

SEE ALL ARTICLES