After a while using Kubernetes, I tried to improve the cluster and try to use all the features from the cloud provider such as the autoscaling groups and multiple availability zones.

The environment

• AWS (The idea is the same for other cloud providers)
• Kubernetes
• Autoscaling groups for the workes for each availability zone (zone-a, zone-b, zone-c).
• Cluster-Autoscaler (CA)

The cluster will scale when:

• There are pods that failed to run in the cluster due to insufficient resources.
• There are nodes in the cluster that have been underutilized for an extended period of time and their pods can be placed on other existing nodes.

I'm working with Airflow and the Kubernetes operator which deploys from Airflow a container to Kubernetes. The pods created by Airflow have ended so the final status of the pod could be Succeeded or Failed. I was looking for to delete the pods created by Airflow but older than 5 days, so we can still read the logs from the pods until they are deleted.

I been playing with the Kubernetes API and I woud like to post and update with the time some examples.

For my case I have a service account which has complete access to a particular namespace, the service account is called demo-user.

To do some test I created a pod and I configured the service account to the pod, so I can get the credentials inside the container and query the API with the service account.

DAG example using KubernetesPodOperator, the idea is run a Docker container in Kubernetes from Airflow every 30 minutes.

Features:

• Scheduled every 30 minutes.
• Set environment variable for the pod RULES.
• Run the pods in the namespace default.
• Mount a volume to the container. It's just an example mounting the /tmp from host.
• Set imagePullSecrets for private Docker registry.

This is just an example about how to keep running a pod without exit, this helps to do troubleshooting or test, this approach is not recommended to use in other cases.

macOS Catalina doesn't support anymore bash and came with zsh, I didn't test this guideline in the macOS Catalina or Big Sur.

macOS comes with a really old bash version. Check the version from the environment variable..

$ echo $BASH_VERSION
3.2.57(1)-release

We can install a newer version of bash via Homebrew package manager, get Homebrew from here https://brew.sh.