Kubernetes multiple zones and autoscaling

July 6, 2020

After a while using Kubernetes, I tried to improve the cluster and try to use all the features from the cloud provider such as the autoscaling groups and multiple availability zones.

The environment

  • AWS (The idea is the same for other cloud providers)
  • Kubernetes
  • Autoscaling groups for the workes for each availability zone (zone-a, zone-b, zone-c).
  • Cluster-Autoscaler (CA)

The cluster will scale when:

  • There are pods that failed to run in the cluster due to insufficient resources.
  • There are nodes in the cluster that have been underutilized for an extended period of time and their pods can be placed on other existing nodes.
Read more

Kubernetes: Delete pods older than X days

February 14, 2020

I'm working with Airflow and the Kubernetes operator which deploys from Airflow a container to Kubernetes. The pods created by Airflow have ended so the final status of the pod could be Succeeded or Failed. I was looking for to delete the pods created by Airflow but older than 5 days, so we can still read the logs from the pods until they are deleted.

Read more

Kubernetes API by examples

February 11, 2020

I been playing with the Kubernetes API and I woud like to post and update with the time some examples.

For my case I have a service account which has complete access to a particular namespace, the service account is called demo-user.

To do some test I created a pod and I configured the service account to the pod, so I can get the credentials inside the container and query the API with the service account.

Read more

Airflow: Kubernetes Operator

November 4, 2019

DAG example using KubernetesPodOperator, the idea is run a Docker container in Kubernetes from Airflow every 30 minutes.

Features:

  • Scheduled every 30 minutes.
  • Set environment variable for the pod RULES.
  • Run the pods in the namespace default.
  • Mount a volume to the container. It's just an example mounting the /tmp from host.
  • Set imagePullSecrets for private Docker registry.
Read more


Kubectl autocompletion in macOS

March 20, 2019

macOS Catalina doesn't support anymore bash and came with zsh, I didn't test this guideline in the macOS Catalina or Big Sur.

macOS comes with a really old bash version. Check the version from the environment variable..

$ echo $BASH_VERSION
3.2.57(1)-release

We can install a newer version of bash via Homebrew package manager, get Homebrew from here https://brew.sh.

Read more