[Infrastructure as Code (IaC)] Pulumi Generate Kubernetes YAML with Golang (Go)

Posted on 2021-04-10 Edited on 2024-09-12 In Infrastructure as Code (IaC) , Cloud Native , Kubernetes (K8S) , Pulumi Views: Word count in article: 6.8k Reading time ≈ 6 mins.

Rendering Kubernetes YAML from Golang (Go)

Pulumi is a Modern Infrastructure as Code (IaC) to create, deploy, and manage infrastructure on any cloud using familiar programming languages and tools.

Pulumi can generate Kubernetes manifests that easily integrate into existing CI/CD workflows with your familiar programming languages.

This article is about how to use Pulumi and Go SDK to manager Namespace and Deployment within Kubernetes (K8S).

Pulumi has excellent support for deploying and updating Kubernetes resources on a cluster, many users have asked for the option to render YAML that they can integrate into existing workflows. pulumi-kubernetes adds the renderYamlToDirectory option, which enables this feature. This option is available in every Pulumi-supported language, including TypeScript/JavaScript, Python, Go, and .NET.

Aside from easily templating configuration across resources, using a familiar programming language allows you to write and consume libraries, and easily mix in infrastructure configuration (e.g., managed database endpoints, object storage, etc.), all in the same program.

Prerequisites

Kubernetes (K8S)
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.

Install Pulumi and Go

Pulumi

Install the Pulumi - https://www.pulumi.com/ CLI.

1 2	# Mac OS X $ brew install pulumi

Go Language Runtime

Install Go - https://golang.org/ .

1 2	# Mac OS X $ brew install go

Configure Kubernetes

By default, Pulumi will look for a kubeconfig file in the following locations, just like kubectl:

The environment variable: $KUBECONFIG,
Or in current user’s default kubeconfig directory: ~/.kube/config

If the kubeconfig file is not in either of these locations, Pulumi will not find it, and it will fail to authenticate against the cluster. Set one of these locations to a valid kubeconfig file, if you have not done so already.

Pulumi New

Create the workspace directory.

1
2
3

$ mkdir -p col-k8s-yaml-gen-example

$ cd col-k8s-yaml-gen-example

Pulumi login into local file system.

1
2
3

$ pulumi login file://.
Logged in to cloudolife as cloudolife (file://.)
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.

Pulumi new a project with go SDK.

$ pulumi new kubernetes-go
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (ccol-k8s-yaml-gen-example) 
project description: (A minimal Kubernetes Go Pulumi program) 
Created project 'ccol-k8s-yaml-gen-example'

stack name: (dev) 
Created stack 'dev'
Enter your passphrase to protect config/secrets: 
Re-enter your passphrase to confirm: 

Enter your passphrase to unlock config/secrets
    (set PULUMI_CONFIG_PASSPHRASE or PULUMI_CONFIG_PASSPHRASE_FILE to remember): 
Installing dependencies...

Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run 'pulumi up'

The above command will create some files within the current directory.

tree .
.
├── Pulumi.dev.yaml
├── Pulumi.yaml
├── go.mod
├── go.sum
└── main.go

See and modify main.go file.

package main

import (
	appsv1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/apps/v1"
	corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/core/v1"
	metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/meta/v1"
	provider "github.com/pulumi/pulumi-kubernetes/sdk/v2/go/kubernetes/providers"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {

	pulumi.Run(func(ctx *pulumi.Context) error {

		provider, _ := provider.NewProvider(ctx, "render-yaml", &provider.ProviderArgs{
            // Choose a directory for the rendered manifests
			RenderYamlToDirectory: pulumi.String("kubernetes"),
		})

		name := "nginx"

		appLabels := pulumi.StringMap{
			"app": pulumi.String("nginx"),
		}

		deployment, err := appsv1.NewDeployment(ctx, name, &appsv1.DeploymentArgs{
			Metadata: metav1.ObjectMetaArgs{
				Name: pulumi.String(name),
			},
			Spec: appsv1.DeploymentSpecArgs{
				Selector: &metav1.LabelSelectorArgs{
					MatchLabels: appLabels,
				},
				Replicas: pulumi.Int(1),
				Template: &corev1.PodTemplateSpecArgs{
					Metadata: &metav1.ObjectMetaArgs{
						Labels: appLabels,
					},
					Spec: &corev1.PodSpecArgs{
						Containers: corev1.ContainerArray{
							corev1.ContainerArgs{
								Name:  pulumi.String("nginx"),
								Image: pulumi.String("nginx"),
							}},
					},
				},
			},
		}, 
        // Specify the Provider.
        pulumi.Provider(provider))
		if err != nil {
			return err
		}

		ctx.Export("name", deployment.Metadata.Elem().Name())

		return nil
	})
}

Pulumi Update

Then, run pulumi update to generate Kubernetes YAML manifests.

1	$ pulumi update

It will generate Kubernetes YAML manifests into kubernetes directory.

$ tree kubernetes/
kubernetes
├── 0-crd
└── 1-manifest
    └── deployment-nginx.yaml

See deployment-nginx.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"app.kubernetes.io/managed-by":"pulumi"},"name":"nginx"},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"image":"nginx","name":"nginx"}]}}}}
  labels:
    app.kubernetes.io/managed-by: pulumi
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

Note that CustomResourceDefinition resources need to be applied first, so they are rendered in a separate subdirectory. (This example doesn’t include any CRDs, so the directory is empty). You could deploy the rendered manifests with kubectl like this:

1 2	$ kubectl apply -f kubernetes/0-crd $ kubectl apply -f kubernetes/1-manifest

Caveats

There are two important caveats to note about YAML rendering support:

The YAML-rendered resources are not created on a Kubernetes cluster, so information that is computed server-side will not be available in your program. For example, a Service will not have IP assignments, so attempting to export these values will not work as usual (i.e., the value will be undefined).
Any Secret values will appear in plaintext in the rendered manifests. This includes any values marked as secret in Pulumi. A warning will be printed for any secret values being rendered to YAML, but it is your responsibility to protect the rendered files.