Automate All the Boring Kubernetes Operations With Python | by Martin Heinz | May, 2022

Learn the way you should use Python’s Kubernetes Consumer library to automate all of the boring Kubernetes duties and operations

Background Picture by Sharon McCutcheon on Unsplash

Kubernetes grew to become a de-facto customary lately, and many people — each DevOps engineers and builders alike — use it every day. Most of the duties we carry out are, nevertheless, the identical, boring, and simple to automate.

Usually, it’s easy sufficient to whip up a fast shell script with a bunch of kubectl instructions, however for extra sophisticated automation duties bash simply is not ok, and also you want the ability of correct languages, equivalent to Python.

So, on this article, we are going to take a look at how one can leverage the Kubernetes Python Consumer library to automate no matter annoying Kubernetes process you is likely to be coping with!

Earlier than we begin taking part in with the Kubernetes shopper, we first must create a playground cluster the place we will safely check issues out. We are going to use KinD (Kubernetes in Docker), which you’ll be able to set up from here.

We are going to use the next cluster configuration:

To create a cluster from the above configuration, you may run:

With the cluster up and working, we additionally want to put in the shopper library (optionally, contained in the digital setting):

To carry out any motion inside our Kubernetes cluster, we first must authenticate.

We are going to use long-lived tokens so we don’t must undergo the authentication circulation repeatedly. Lengthy-lived tokens might be created by making a ServiceAccount:

Utilizing a service account additionally has the profit that it’s not tied to any single particular person, which is all the time preferable for automation functions.

The token from the output above might be then utilized in requests:

We’re authenticated now, however not approved to do a lot of something. Due to this fact, subsequent, we have to create a Function and bind it to the ServiceAccount so we will carry out actions on assets. Right here’s the code:

The above offers our service account permission to carry out any motion on pods, restricted to default namespace.

It is best to all the time preserve your roles very slender and particular, however taking part in round in KinD, it is sensible to use a cluster-wide admin position, as you may see beneath:

To get a greater understanding of what’s kubectl and likewise the shopper doing beneath the hood, we are going to begin with uncooked HTTP requests utilizing curl.

The best option to discover out what requests are made beneath the hood is to run the specified kubectl command with -v 10. This may output full curl instructions, that are proven beneath:

The output with loglevel 10 might be very verbose, however someplace in there, you will see the above curl command.

Add a Bearer token header within the above curl command together with your long-lived token, and it is best to be capable to carry out the identical actions as kubectl, equivalent to the next:

In case there’s a request physique wanted, search for which fields have to be included within the request. For instance, when making a Pod, we will use the API described here. Once we do, we get the next request:

Check with the Kubernetes API reference for object attributes. Moreover, you too can view the OpenAPI definition with this command:

Interacting with Kubernetes straight utilizing REST API is likely to be a bit clunky, however there are conditions the place it’d make sense to make use of it. Some conditions embrace interacting with APIs that don’t have any equal kubectl command once you’re utilizing a unique distribution of Kubernetes — equivalent to OpenShift — which exposes further APIs not coated by both kubectl or shopper SDK.

Shifting onto the Python shopper itself now. We have to undergo the identical step as with kubectl or curl. Step one is authentication, which is proven beneath:

First, we outline a configuration object, which tells the shopper that we are going to authenticate utilizing the Bearer token. Contemplating that our KinD cluster doesn’t use SSL, we disable it in the true cluster. Nonetheless, it is best to by no means do this.

To check out the configuration, we use list_namespaced_pod methodology of API shopper to get all pods within the default namespace, after which we print out their title, namespace and IP.

Now, for a extra lifelike process, let’s create a Deployment beneath:

Along with creating the Deployment, we additionally look forward to its pods to turn out to be accessible. We do this by querying the Deployment standing and checking various accessible replicas.

Additionally, discover the sample in operate names, equivalent to create_namespaced_deployment. To make it extra apparent, let us take a look at a pair extra:

  • replace_namespaced_cron_job
  • patch_namespaced_stateful_set
  • list_namespaced_horizontal_pod_autoscaler
  • read_namespaced_daemon_set
  • read_custom_resource_definition

All of those are within the format operation_namespaced_resource or simply operation_resource for international assets. They are often moreover suffixed with _status or _scale for strategies that carry out operations on useful resource standing, equivalent to read_namespaced_deployment_status or useful resource scale equivalent to patch_namespaced_stateful_set_scale.

One other factor to focus on is that within the above instance, we carried out the actions utilizing shopper.AppsV1Api which permits us to work with all of the assets that belong to apiVersion: apps/v1. If we wished to make use of CronJob, we’d as an alternative select BatchV1Api (which is apiVersion: batch/v1 in YAML format), or for PVCs, we’d select CoreV1Api due to apiVersion: v1 . You get the gist.

As you may think about, that’s numerous features to select from. Fortunately, all of them are listed in docs, and you may click on on any one in every of them to get an instance of its utilization.

Past fundamental CRUD operations, it’s additionally doable to observe objects for modifications repeatedly. The apparent alternative is to observe Occasions:

Right here we selected to observe occasions in default namespace. We take the primary 10 occasions after which shut the stream. If we wished to watch the assets repeatedly, we’d simply take away the timeout_seconds and the w.cease() name.

Within the first instance, you noticed that we used plain Python dict to outline the Deployment object, which we handed to the shopper. Alternatively, we will use a extra OOP type through the use of API Fashions (lessons) offered by the library, proven beneath:

Attempting to determine which mannequin it is best to use for every argument is a shedding battle, and it’s robust. When creating assets like these proven above, it is best to all the time use documentation for models and traverse the hyperlinks as you create the person subobjects to determine what values/varieties are anticipated in every discipline.

It is best to now have a fundamental concept about how the shopper works, so let’s check out some helpful examples and snippets that may make it easier to automate every day Kubernetes operations.

A quite common factor you may need to carry out is a Deployment rollout — often achieved with kubectl rollout restart. Nonetheless, there’s no API to do that. The way kubectl does it is by updating Deployment Annotations. Extra particularly, setting kubectl.kubernetes.io/restartedAt to the present time. This works as a result of any change made to Pod spec causes a restart.

If we need to carry out a restart utilizing the Python shopper, we have to do the next:

One other widespread operation is scaling a Deployment. Thankfully, this one has an API operate we will use, which you’ll be able to see beneath:

For troubleshooting functions, it typically is sensible to exec right into a Pod, have a look round, and probably seize the setting variable to confirm the proper configuration. Right here’s the code for that:

The snippet above additionally lets you run entire shell scripts if want be.

So, let’s say you need to apply a Taint onto a node that has some subject. We are able to give attention to cluster administration-oriented duties. Since there’s no direct API for Node Taints, we will discover a means. Right here’s the code to assist us:

You may additionally need to monitor cluster useful resource utilization to automate cluster scaling. Often, you’d use kubectl prime to get the data interactively, however with the shopper library, you are able to do the next:

The above instance assumes that you’ve metrics-server put in in your cluster. You may run kubectl prime to confirm that. Use the remark within the snippet to put in it if you happen to’re working with KinD.

Final however not least, if in case you have a bunch of YAML or JSON recordsdata you need to use to deploy or modify objects in your cluster, otherwise you need to export and backup what you’ve created with the shopper, there’s a simple means. Right here’s how one can convert YAML/JSON recordsdata to Kubernetes objects and again to recordsdata once more:

The primary option to convert an present object right into a Python dictionary (JSON) is to make use of sanitize_for_serialization which produces uncooked output with all of the generated/default fields. A greater possibility is to make use of utility strategies of the kopf library, which is able to take away all pointless fields. From there, it is easy sufficient to transform a dictionary into a correct YAML or JSON file.

If you wish to reverse this course of — go from the dictionary to a Consumer Object Mannequin — we will use deserialize methodology of API Consumer. This methodology, nevertheless, expects its argument to have a knowledge attribute, so we cross it a container class occasion with such an attribute.

If you have already got YAML recordsdata that you simply’d like to make use of with the Python shopper, you should use the utility operate, kubernetes.utils.create_from_yaml.

To get a whole overview of all of the library options, I like to recommend you check out the examples directory within the repository.

I’d additionally encourage you to look by means of the problems within the library repository, because it has numerous nice examples of shopper utilization, equivalent to processing events in parallel or watching ConfigMaps for updates.

The Python shopper library accommodates lots of of features, so it’s troublesome to cowl each little characteristic or use case there may be. Most of them comply with a standard sample which ought to make the library’s utilization fairly pure after a few minutes.

In case you’re on the lookout for extra examples past what was proven and referenced above, I like to recommend exploring different well-liked instruments that make use of the Python Kubernetes shopper, such because the library for creating Kubernetes operators. I additionally discover it very helpful to try exams of the library itself, because it showcases its supposed utilization. Right here’s a wonderful client test suite to check.

Need to Join?This text was initially posted at martinheinz.dev

More Posts