Write Your Kubernetes Infrastructure as Go Code — Using Custom Resource Definitions With Cdk8s | by Abhishek Gupta | Jul, 2022

Utilizing CRDs as API

Picture by Ihor Dvoretskyi on Unsplash

cdk8s (Cloud Growth Equipment for Kubernetes) is an open-source framework (a part of CNCF) utilizing which you’ll be able to outline your Kubernetes purposes with common programming languages (as a substitute of yaml). A number of the earlier blogs on this matter lined the getting began expertise and utilizing cdk8s-plus library to additional enhance upon the core cdk8s library options. We’re going to proceed and push cdk8s even additional.

This weblog publish will display how you should use Kubernetes Custom Resource Definitions with cdk8s. We are going to begin off with a easy Nginx instance after which you’ll use the mix of Strimzi venture CRDs together with Go and cdk8s to outline and deploy a Kafka cluster on Kubernetes!

I’m assuming that you just’ve have some information of Kubernetes Customized Useful resource Definitions and have in all probability even used a couple of within the type of Operators. If not, that’s okay! The Kubernetes documentation covers it fairly properly. You possibly can all the time consult with it, come again right here and comply with alongside!

cdk8s permits you to use Kubernetes API objects instantly in your code, with out having to import particular person Go shopper packages, all because of cdk8s import. (additionally talked about within the “Wait, what in regards to the Kubernetes API dependencies??” part of a previous blog post). However you can even use it for Customized Useful resource Definitions! Let’s examine this in motion.

Be sure you have Go (v1.16 or above) and cdk8s CLI put in. Additionally, you must have entry to a Kubernetes cluster. For studying and experimentation, I might suggest utilizing a single-node cluster operating domestically – reminiscent of minikube, kind, and so forth.

I usually use minikube, so organising a cluster is so simple as minikube begin

To put in cdk8s CLI:

You possibly can select from the beneath choices:

brew set up cdk8s
npm set up -g cdk8s-cli
yarn international add cdk8s-cli

Though this weblog publish will present step-by-step directions, you possibly can all the time consult with the whole code on Github

cdk8s makes it very easy for you get began and bootstrap your utility. You needn’t guess and determine the right way to construction your venture, setup dependencies and so forth. because the cdk8s init command does it for you!

cdk8s init go-app#output
Your cdk8s Go venture is prepared! cat assist Prints this message
cdk8s synth Synthesize k8s manifests to dist/
cdk8s import Imports k8s API objects to "imports/k8s"
kubectl apply -f dist/

Replace the generate go.mod file, and substitute it with the next – that is to make issues easier for you.

Be at liberty to make use of the most recent model of the modules if wanted.

module cdk8s-crdgo 1.16require (
github.com/aws/constructs-go/constructs/v10 v10.1.42
github.com/aws/jsii-runtime-go v1.61.0
github.com/cdk8s-team/cdk8s-core-go/cdk8s/v2 v2.3.34

To begin with, let’s work with a very (actually!) easy Customized Useful resource Definition.

I’m going to make use of a pattern CRD from the Kubernetes example. To be sincere, it doesn’t actually do something. However, since we’re simply getting began, this could suffice!

First, set up/register the CRD useful resource itself:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/sample-controller/master/artifacts/examples/crd.yaml

Verify whether or not the CRD was put in:

kubectl get crd# output
foos.samplecontroller.k8s.io 2022-07-08T09:28:46Z
kubectl get foos.samplecontroller.k8s.io#output (as anticipated)
No assets present in default namespace.

So, we simply put in a CRD with the identify foos.samplecontroller.k8s.io and kind Foo. Its potential to create an occasion of this utilizing yaml… however…

We’re right here to write down Go code!

To do this, first import the CRD as an API utilizing cdk8s – this may robotically create the corresponding Go API representations (structs and so forth.):

cdk8s import https://raw.githubusercontent.com/kubernetes/sample-controller/master/artifacts/examples/crd.yaml

Examine the imports listing, a further folder ought to have been created.

└── samplecontrollerk8sio
├── inner
│ └── sorts.go
├── jsii
│ ├── jsii.go
│ └── samplecontrollerk8sio-0.0.0.tgz
├── samplecontrollerk8sio.go
├── samplecontrollerk8sio.init.go
└── model

We are able to now use the CRD similar to some other Kubernetes useful resource/API (like Deployment) and import it within the cdk8s Go code. Create a brand new file known as foo.go and replica the next code:

package deal majorimport (
sort FooChartProps struct
func NewFooChart(scope constructs.Assemble, id string, props *FooChartProps) cdk8s.Chart
var cprops cdk8s.ChartProps
if props != nil
cprops = props.ChartProps

chart := cdk8s.NewChart(scope, jsii.String(id), &cprops)
samplecontrollerk8sio.NewFoo(chart, jsii.String("foo1"), &samplecontrollerk8sio.FooPropsSpec: &samplecontrollerk8sio.FooSpecDeploymentName: jsii.String("foo1-dep"), Replicas: jsii.Quantity(2)) return chart

See how we created an occasion of samplecontrollerk8sio.Foo:

  • Imported the autogenerated CRD API from the cdk8s-crd/imports/samplecontrollerk8sio package deal,
  • Used the NewFoo perform and offered the metadata by way of FooProps

Exchange the contents of major.go with the next:

package deal majorimport (
sort MyChartProps struct
func major()
app := cdk8s.NewApp(nil)
NewFooChart(app, "FooApp", nil)

All we is embody the Chart that we outlined simply now (in foo.go) and embody it within the cdk8s App.

To create the Foo useful resource…

Run cdk8s synth – this may lead to a manifest within the dist folder:

apiVersion: samplecontroller.k8s.io/v1alpha1
sort: Foo
deploymentName: foo1-dep
replicas: 2
identify: fooapp-foo1-c80094ac

To create it in Kubernetes:

kubectl apply -f dist

You possibly can affirm by operating :

kubectl get foo
kubectl get foos.samplecontroller.k8s.io

To introspect additional, you should use the identify of the created useful resource e.g. kubectl describe foo/fooapp-foo1-c80094ac

Alright, now that you just’ve seen a easy instance, we are able to transfer on to one thing barely extra superior.

Strimzi is an open-source CNCF venture and one in all my private favourites! In case you don’t find out about Strimzi, that’s okay. It’s sufficient to know that it gives a method to run an Apache Kafka on Kubernetes with the assistance of Customized Useful resource Definitions and corresponding Operators for parts reminiscent of Kafka cluster, Kafka Join matter, customers, Kafka Mirror and so forth.

Here’s a high-level diagram of how the completely different Strimzi parts work together. Since a Strimzi deep-dive is out of scope, I might suggest that you just refer its (glorious!) documentation for particulars.


As earlier than, we have to first set up the CRD itself (you can even consult with the Strimzi Quickstart)

kubectl create namespace kafka
kubectl create -f 'https://strimzi.io/set up/newest?namespace=kafka' -n kafka
# anticipate the Operator Pod to start out up (Operating)
kubectl get pod -n kafka --watch

You too can examine the Operator logs utilizing kubectl logs deployment/strimzi-cluster-operator -n kafka -f

Every supported Kafka element (cluster, join, consumer and so forth.) has a corresponding Customized Useful resource Definition — for the needs of this weblog publish, we’ll simply use the Kafka cluster and matter CRDs. Let’s import them as an API:

cdk8s import https://uncooked.githubusercontent.com/strimzi/strimzi-kafka-operator/major/set up/cluster-operator/040-Crd-kafka.yamlcdk8s import kafkatopic:=https://uncooked.githubusercontent.com/strimzi/strimzi-kafka-operator/major/set up/cluster-operator/043-Crd-kafkatopic.yaml

Word that I’ve prepended kafkatopic to the module identify for Kafka matter CRD

Examine the imports folder – it is best to see two extra folders named kafkastrimziio and kafkatopic_kafkastrimziio.

Time for some Go code, once more:

Create a kafka_strimzi.go file and replica the code from Github repo:

Or you can even merely do that: curl -o kafka.go https://raw.githubusercontent.com/abhirockzz/cdk8s-for-go-developers/master/part3-crd/kafka_strimzi.go

I’ll stroll you thru the essential elements of the code right here. Begin with the NewKafkaChart perform that creates a brand new Chart.

func NewKafkaChart(scope constructs.Assemble, id string, props *KafkaChartProps) cdk8s.Chart {
//.... ommitted for brevity
chart := cdk8s.NewChart(scope, jsii.String(id), &cprops)

See how the Kafka cluster is outlined utilizing kafkastrimziio.KafkaProps struct (for a deep dive into every of those parts you possibly can consult with Strimzi documentation). We specify the Kafka model, variety of nodes/replicas (we’ll follow a single node duplicate) the right way to expose the cluster and so forth.

Spec: &kafkastrimziio.KafkaSpec
Kafka: &kafkastrimziio.KafkaSpecKafka
Model: jsii.String("3.2.0"),
Replicas: jsii.Quantity(1),
Listeners: &[]*kafkastrimziio.KafkaSpecKafkaListeners

Identify: jsii.String("plain"),
Port: jsii.Quantity(9092),
Kind: kafkastrimziio.KafkaSpecKafkaListenersType_INTERNAL,
Tls: jsii.Bool(false),

Then we add the required config for the Kafka cluster (in-line with the truth that now we have a single node cluster solely) in addition to storage (ephemeral storage will work for this instance).

Config: map[string]interface
"offsets.matter.replication.issue": 1,
"transaction.state.log.replication.issue": 1,
"transaction.state.log.min.isr": 1,
"default.replication.issue": 1,
"min.insync.replicas": 1,
"inter.dealer.protocol.model": "3.2",
Storage: &kafkastrimziio.KafkaSpecKafkaStorage
Kind: kafkastrimziio.KafkaSpecKafkaStorageType_EPHEMERAL,

Lastly, we configure Zookeeper in addition to the Entity operator that handles Kafka matters (in addition to customers, though we don’t use it right here)

Zookeeper: &kafkastrimziio.KafkaSpecZookeeper
Replicas: jsii.Quantity(1),
Storage: &kafkastrimziio.KafkaSpecZookeeperStorage
Kind: kafkastrimziio.KafkaSpecZookeeperStorageType_EPHEMERAL,
EntityOperator: &kafkastrimziio.KafkaSpecEntityOperator
TopicOperator: &kafkastrimziio.KafkaSpecEntityOperatorTopicOperator,

To wire it up, replace the major.go file:

func major() 
app := cdk8s.NewApp(nil)
//NewFooChart(app, "FooApp", nil)
NewKafkaChart(app, "KafkaApp", nil)

To create a Kafka cluster utilizing the CRD…

Observe the same old workflow:

# generate manifest (examine it in dist folder)
cdk8s synth
# apply it (notice the kafka namespace)
kubectl apply -f dist/ -n kafka

Look ahead to the assets to be created:

KAFKA_CRD_INSTANCE_NAME=$(kubectl get kafka -n kafka -o=jsonpath='.objects[0].metadata.identify')
kubectl wait kafka/$KAFKA_CRD_INSTANCE_NAME --for=situation=Prepared --timeout=300s -n kafka

As soon as all of the Kafka cluster assets are created, it is best to see the next message — kafka.kafka.strimzi.io/<identify of your Kafka CRD occasion> situation met. The Kafka cluster is now prepared and we are able to take a look at it utilizing the nice outdated Kafka CLI-based producer and shopper (directions in Strimzi quickstart).

BOOSTRAP_SERVER=$(kubectl get kafka -n kafka -o=jsonpath='.objects[0].metadata.identify')-kafka-bootstrapkubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:0.29.0-kafka-3.2.0 --rm=true --restart=By no means -- bin/kafka-console-producer.sh --bootstrap-server $BOOSTRAP_SERVER:9092 --topic test-topickubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:0.29.0-kafka-3.2.0 --rm=true --restart=By no means -- bin/kafka-console-consumer.sh --bootstrap-server $BOOSTRAP_SERVER:9092 --topic test-topic --from-beginning

That’s all for now!

You discovered the right way to mix Kubernetes Customized Useful resource definition with cdk8s. That is actually highly effective and means that you may proceed to make use of code (on this case, written in Go) to outline built-in Kubernetes assets (like Deployments and so forth.) in addition to Customized assets!

Did you want what you tried?

Effectively, you possibly can proceed studying!

A few options:

  1. You possibly can strive updating the prevailing code so as to add a Deployment useful resource that refers to a Kafka shopper app (you need to write it and package deal it as a Docker container first) and may entry the Kafka cluster you created. Discover how one can get the connectivity parameters..
  2. The Kafka cluster we created was configured to have Inside entry solely. Discover choices to show it externally (consult with Strimzi documentation) and replace the code to implement that (needs to be a small change). Which Kubernetes objects will probably be affected by it?

Joyful coding!

More Posts