Skip to content

Set Up LocaL Development Envrionment

Overview

You have a kubernetes cluster, and you want to develop a mutating webhook.

You can develop the code locally and deploy it into the cluster to test and debug the codes in the container.

However, if any bugs are found, you need to:

  • edit the codes
  • rebuild the image
  • deploy it into the cluster
  • and test again.

Quite annoying.🤬🤬

Instead of that, a better choice would be:

  • run a server (flask or fastapi) locally on your computer
  • the kubernetes sends the admission review requests to this server.

Then you can test, debug and update codes totally in your local environment😍.

There are two ways to achieve this:

Warning

Webhooks are required to support TLS. You need setup the certificates all yourself!

  • Another way would be the approach in this tutorial.

We setup all the kubernetes resources just as if we are really deploying a production-ready webhook and the certificates are automatically managed by the cert-manager.

Just one difference, instead of deploying the webhook container with a webhook image, we use an nginx image instead as a reverse proxy and pass the requests to our server in the local envrionment.

The benefit of this is: after you have developed and tested the webhook codes, you can build the image and simply replace the nginx deployment, the webhook will work in no time. And you can switch back to local development at anytime you want.

Prepare YAMLs

Let's look at the YAMLs

apiVersion: v1
kind: Namespace
metadata:
  name: webhook-test
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: webhook-cert
  namespace: webhook-test
spec:
  commonName: webhook-service.webhook-test.svc
  dnsNames:
  - webhook-service.webhook-test.svc
  issuerRef:
    kind: Issuer
    name: selfsigned-issuer
  secretName: webhook-service-cert
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
  namespace: webhook-test
spec:
  selfSigned: {}

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  annotations:
    cert-manager.io/inject-ca-from: webhook-test/webhook-cert
  name: webhook-test
webhooks:
- admissionReviewVersions:
  - v1beta1
  clientConfig:
    caBundle: Cg==
    service:
      name: webhook-service
      namespace: webhook-test
      path: /mutate-pods
  failurePolicy: Fail
  name: webhook-service.pod.mutator
  objectSelector:
    matchLabels:
      mutate: "true"
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - pods
  sideEffects: None

apiVersion: v1
kind: Service
metadata:
  name: webhook-service
  namespace: webhook-test
spec:
  ports:
  - protocol: TCP
    port: 443
    targetPort: 443
  selector:
    app: webhook-test

❗️❗️❗️Replace the IP with your own computer's.

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: webhook-test
  name: nginx-conf
data:
  nginx.conf: |
    user nginx;
    worker_processes  3;
    events {
      worker_connections  10240;
    }
    http {
      ssl_certificate /etc/ssl/tls.crt;
      ssl_certificate_key /etc/ssl/tls.key;
      ssl_protocols TLSv1.2 TLSv1.3;
      server {
          listen 443 ssl;
          location / {
              proxy_pass http://192.168.10.10:8000;
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header Host $http_host;
              proxy_set_header X-Real-IP $remote_addr;
              proxy_redirect off;
          }
      }
    }

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: webhook-test
  name: webhook-reverse-proxy
  labels:
    app: webhook-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: webhook-test
  template:
    metadata:
      labels:
        app: webhook-test
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 443
        volumeMounts:
        - mountPath: /etc/ssl
          name: cert
          readOnly: true
        - name: nginx-conf
          mountPath: /etc/nginx/nginx.conf
          subPath: nginx.conf
          readOnly: true
      terminationGracePeriodSeconds: 5
      volumes:
      - name: cert
        secret:
          defaultMode: 420
          secretName: webhook-service-cert
      - name: nginx-conf
        configMap:
          name: nginx-conf
          items:
            - key: nginx.conf
              path: nginx.conf

Save all the YAMLs in a directory webhook.

Install

$ kubectl apply -f webhook.yaml
namespace/webhook-test created
certificate.cert-manager.io/webhook-cert created
issuer.cert-manager.io/selfsigned-issuer created
mutatingwebhookconfiguration.admissionregistration.k8s.io/webhook-test created
service/webhook-service created
configmap/nginx-conf created
deployment.apps/webhook-reverse-proxy created

Uninstall

$ kubectl delete -f webhook/
namespace "webhook-test" deleted
certificate.cert-manager.io "webhook-cert" deleted
issuer.cert-manager.io "selfsigned-issuer" deleted
mutatingwebhookconfiguration.admissionregistration.k8s.io "webhook-test" deleted
service "webhook-service" deleted
configmap "nginx-conf" deleted
deployment.apps "webhook-reverse-proxy" deleted