Deploy Operator into Cluster¶

1. Flaws¶

Although our operator works, there are still flaws:

deployment's status is not synchronized to nginx resource's status.
modifications/deletion during operator downtime are not processed.
not async

These are not hard to achieve, however, to keep it simple here, I won't implement these functions in this tutorial.

Let's now focus on the whole process of creating and deploying an operator, then you can think about improving it and release a new version!

2. Operator Codes¶

operator-in-cluster.py

Save

Save the following codes as operator-in-cluster.py. There is a small change: using load_incluster_config() instead of load_kube_config().

import logging
import datetime
from kubernetes import client, config, watch
from nginx import Nginx


logging.basicConfig(level=logging.INFO)
logger = logging.getLogger('underneathall')
# load config
config.load_incluster_config()


def handle_event(event):
    try:
        name = event['object']['metadata']['name']
    except Exception as exc:
        logger.error('Invalid event received: %s, %s', str(event), str(exc))
        return False
    logger.info(
        'Start processing event: %s %s',
        event['object']['metadata']['name'], event.get('type'))
    logger.debug('event: %s', str(event))
    nginx = Nginx(
        name=name,
        namespace=event['object']['metadata']['namespace'],
        replicas=event['object']['spec']['replicas']
    )
    nginx.sync(event_type=event['type'])
    return True


if __name__ == '__main__':
    # create a custom resource api instance
    crapi = client.CustomObjectsApi()
    # create a kubernetes watch instance
    watch = watch.Watch()
    # list custom resource function
    list_cr = client.CustomObjectsApi().list_cluster_custom_object
    # watch the stream and print out the events' info
    logger.info("Operator started.")
    for e in watch.stream(list_cr, 'stable.underneathall.com', 'v1', 'nginx'):
        try:
            handle_event(e)
        except Exception as exc:
            logger.error('Event handling failed: %s', e)

3. Build an Image¶

Dockerfile

Run

In the same folder with our operator codes (.py files), run the following command to save the Dockerfile.

cat <<DOCKERFILE> Dockerfile
FROM python:3.9.1-slim-buster

RUN pip install --no-cache-dir kubernetes==12.0.1
RUN mkdir -p /opt/underneathall
WORKDIR /opt/underneathall
COPY nginx.py /opt/underneathall/nginx.py
COPY operator-in-cluster.py /opt/underneathall/operator.py
RUN touch __init__.py

CMD ["python", "operator.py"]
DOCKERFILE

Now, you should have a folder structure like below

Folder Structure

.
├── Dockerfile
├── __init__.py
├── nginx.py
└── operator-in-cluster.py

Build Image

Build

In the same folder, run the following command to build the image.

docker build -t underneathall/nginx-operator:v1 .

Using containerd?

If you are using microk8s as I do, you need to save the image from docker, and then import the image into containerd.

SaveImport

docker save underneathall/nginx-operator:v1 > underneathall-nginx-operator-v1.tar

ctr image import underneathall-nginx-operator-v1.tar

or if you are using microk8s:

microk8s ctr image import underneathall-nginx-operator-v1.tar

3. Role, Service Account, Role Binding¶

Role, Service Account, Role Binding

sa-role-rolebinding.yaml

Save the following codes as sa-role-rolebinding.yaml. We will use it to setup the service account with corresponding permissions for our operator.

Also we will create a new namespace underneathall to deploy our operator.

apiVersion: v1
kind: Namespace
metadata:
name: underneathall
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: underneathall
namespace: underneathall
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: underneathall
rules:
- apiGroups: ["apps"]
resources: ["deployment"]
verbs: ["*"]
- apiGroups: ["stable.underneathall.com"]
resources: ["nginx"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: underneathall
subjects:
- kind: ServiceAccount
    name: underneathall
    namespace: underneathall
roleRef:
kind: ClusterRole
name: underneathall
apiGroup: rbac.authorization.k8s.io

Create Namespace, Service Account, Role, Role Binding

RunOutput

kubectl apply -f sa-role-rolebinding.yaml

namespace/underneathall created
serviceaccount/underneathall created
clusterrole.rbac.authorization.k8s.io/underneathall created
clusterrolebinding.rbac.authorization.k8s.io/underneathall created

4. Create the Operator¶

Deployment

operator.yaml

Save the following codes as operator.yaml.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: underneathall-nginx-operator
  namespace: underneathall
  labels:
    app: underneathall-nginx-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: underneathall-nginx-operator
  template:
    metadata:
      labels:
        app: underneathall-nginx-operator
    spec:
      containers:
      - name: nginx-operator
        image: underneathall/nginx-operator:v1

Create the Operator

CreateVerify

Run

kubectl apply -f operator.yaml

Output

deployment.apps/underneathall-nginx-operator created

Verify the operator is created successfully by checking if the pod is running.

Run

kubectl -n underneathall get po

Output

NAME                                            READY   STATUS    RESTARTS   AGE
underneathall-nginx-operator-5fbb56b57d-gccvz   1/1     Running   0          10s

Now the operator is deployed successfully.

5. Create, Update, and Delete Nginx¶

Let's verify our operator is running normally.

Create

Create Nginx YAMLCreatePods

Save

cat <<NGINX> nginx-1.yaml
apiVersion: "stable.underneathall.com/v1"
kind: Nginx
metadata:
  name: underneathall-nginx-1
  namespace: default
spec:
  replicas: 3
NGINX

Run

kubectl apply -f nginx-1.yaml

Output

nginx.stable.underneathall.com/underneathall-nginx-1 created

Run

kubectl get po

Output

NAME                                            READY   STATUS    RESTARTS   AGE
underneathall-nginx-1-deploy-75ddffcbbf-jd27z   1/1     Running   0          7s
underneathall-nginx-1-deploy-75ddffcbbf-b9lrv   1/1     Running   0          7s
underneathall-nginx-1-deploy-75ddffcbbf-zljds   1/1     Running   0          7s

Update

Update Nginx YAMLUpdatePods

Save

cat <<NGINX> nginx-1.yaml
apiVersion: "stable.underneathall.com/v1"
kind: Nginx
metadata:
  name: underneathall-nginx-1
  namespace: default
spec:
  replicas: 1
NGINX

Run

kubectl apply -f nginx-1.yaml

Output

nginx.stable.underneathall.com/underneathall-nginx-1 configured

Run

kubectl get po

Output

NAME                                            READY   STATUS    RESTARTS   AGE
underneathall-nginx-1-deploy-75ddffcbbf-b9lrv   1/1     Running   0          2m32s

Delete

DeletePods

Run

kubectl delete -f nginx-1.yaml

Output

nginx.stable.underneathall.com "underneathall-nginx-1" deleted

Run

kubectl get po

Output

No resources found in default namespace.

Operator Logs

RunOutput

kubectl -n underneathall logs underneathall-nginx-operator-5fbb56b57d-gccvz --timestamps

2021-01-20T19:55:08.344091023+08:00 INFO:underneathall:Operator started.
2021-01-20T20:08:17.848875859+08:00 INFO:underneathall:Start processing event: underneathall-nginx-1 ADDED
2021-01-20T20:10:13.733721501+08:00 INFO:underneathall:Start processing event: underneathall-nginx-1 MODIFIED
2021-01-20T20:11:25.435068245+08:00 INFO:underneathall:Start processing event: underneathall-nginx-1 DELETED

Deploy Operator into Cluster¶

1. Flaws¶

2. Operator Codes¶

3. Build an Image¶

3. Role, Service Account, Role Binding¶

4. Create the Operator¶

5. Create, Update, and Delete Nginx¶

Congratulations! You have successfully deployed your own operator!¶