[Kubernetes (K8S)] Helm install kube-fledged to create and manage caches of container images directly on the worker nodes within Kubernetes (K8S)

kube-fledged

kube-fledged is a kubernetes operator for creating and managing a cache of container images directly on the worker nodes of a kubernetes cluster. It allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pulled). As a result, application pods start almost instantly, since the images need not be pulled from the registry.

kube-fledged provides CRUD APIs to manage the lifecycle of the image cache, and supports several configurable parameters to customize the functioning as per one’s needs.

This article is about how to use Helm to install kube-fledged on Kubernetes (K8S).

Use cases

• Applications that require rapid start-up. For e.g. an application performing real-time data processing needs to scale rapidly due to a burst in data volume.

• Serverless Functions since they need to react immediately to incoming events.

• IoT applications that run on Edge devices, because the network connectivity between the edge device and image registry would be intermittent.

• If images need to be pulled from a private registry and everyone cannot be granted access to pull images from this registry, then the images can be made available on the nodes of the cluster.

• If a cluster administrator or operator needs to roll-out upgrades to an application and wants to verify before-hand if the new images can be pulled successfully.

How it works

Kubernetes allows developers to extend the kubernetes api via Custom Resources. kube-fledged defines a custom resource of kind “ImageCache” and implements a custom controller (named kubefledged-controller). kubefledged-controller does the heavy-lifting for managing image cache. Users can use kubectl commands for creation and deletion of ImageCache resources.

kubefledged-controller has a built-in image manager routine that is responsible for pulling and deleting images. Images are pulled or deleted using kubernetes jobs. If enabled, image cache is refreshed periodically by the refresh worker. kubefledged-controller updates the status of image pulls, refreshes and image deletions in the status field of ImageCache resource.

For more detailed description, go through kube-fledged’s design proposal.

Configuration Flags for Kubefledged Controller

• --image-pull-deadline-duration: Maximum duration allowed for pulling an image. After this duration, image pull is considered to have failed. default 5m

• --image-cache-refresh-frequency: The image cache is refreshed periodically to ensure the cache is up to date. Setting this flag to 0s will disable refresh. default 15m

• --image-pull-policy: Image pull policy for pulling images into and refreshing the cache. Possible values are IfNotPresent and Always. Default value is IfNotPresent. Image with no or :latest tag are always pulled.

• --stderrthreshold: Log level. set the value of this flag to INFO

Prerequisites

• Kubernetes (K8S)
  Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.

  For more information about installing and using Kubernetes (K8s), see the Kubernetes (K8s) Docs.

• Helm
  Helm is the best way to find, share, and use software built for Kubernetes.

  # Mac OS X
  $ brew install helm

  For more information about installing and using Helm, see the Helm Docs.

Usages

values.yaml

Edit values.yaml and replace content within {{ }}.

# values.yaml

Install Release

Helm install kube-fledged into kube-fledged namespace.

# crate namespace:
$ kubectl create namespace kube-fledged

# Add the Stable Helm repository:
$ helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/

# Update your local Helm chart repository cache:
$ helm repo update

# To install Helm chart:
$ helm install kube-fledged kubefledged-charts/kube-fledged -n kube-fledged -f values.yaml

See pods about kube-fledged.

$ kubectl get pods -n kube-fledged
NAME                                           READY   STATUS    RESTARTS   AGE
kube-fledged-controller-577fdf7fc7-l77cz       1/1     Running   0          33h
kube-fledged-webhook-server-687974b98c-zbhws   1/1     Running   0          33h

How to use

kube-fledged provides APIs to perform CRUD operations on image cache. These APIs can be consumed via kubectl or curl

Create image cache

Refer to sample image cache manifest in deploy/kubefledged-imagecache.yaml. Edit it as per your needs before creating image cache. If images are in private repositories requiring credentials to pull, add imagePullSecrets to the end.

imagePullSecrets:
- name: myregistrykey

Create the image cache using kubectl. Verify successful creation

$ kubectl create -f deploy/kubefledged-imagecache.yaml

$ kubectl get imagecaches -n kube-fledged

View the status of image cache

Use following command to view the status of image cache in json format.

$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Add/remove images in image cache

Use kubectl edit command to add/remove images in image cache. The edit command opens the manifest in an editor. Edit your changes, save and exit.

$ kubectl edit imagecaches imagecache1 -n kube-fledged

$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Refresh image cache

kube-fledged supports both automatic and on-demand refresh of image cache. Auto refresh is enabled using the flag
--image-cache-refresh-frequency:. To request for an on-demand refresh, run the following command:-

$ kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=

Delete image cache

Before you could delete the image cache, you need to purge the images in the cache using the following command. This will remove all cached images from the worker nodes.

$ kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/purge-imagecache=

View the status of purging the image cache. If any failures, such images should be removed manually or you could decide to leave the images in the worker nodes.

$ kubectl get imagecaches imagecache1 -n kube-fledged -o json

Finally delete the image cache using following command.

$ kubectl delete imagecaches imagecache1 -n kube-fledged

Uninstall Release

Destroy release created by Helm.

$ helm uninstall kube-fledged -n kube-fledged

References

[1] senthilrch/kube-fledged: A kubernetes operator for creating and managing a cache of container images directly on the cluster worker nodes, so application pods start almost instantly - https://github.com/senthilrch/kube-fledged

[2] Kubernetes Getting Started | Pulumi - https://www.pulumi.com/docs/get-started/kubernetes/

[3] Kubernetes - https://kubernetes.io/

[4] Helm - https://helm.sh/