Kubernetes: Delete pods older than X days

February 14, 2020

I'm working with Airflow and the Kubernetes operator which deploys from Airflow a container to Kubernetes. The pods created by Airflow have ended so the final status of the pod could be Succeeded or Failed. I was looking for to delete the pods created by Airflow but older than 5 days, so we can still read the logs from the pods until they are deleted.

Read more

Kubernetes API by examples

February 11, 2020

I been playing with the Kubernetes API and I woud like to post and update with the time some examples.

For my case I have a service account which has complete access to a particular namespace, the service account is called demo-user.

To do some test I created a pod and I configured the service account to the pod, so I can get the credentials inside the container and query the API with the service account.

Read more

Kops: Kubernetes cluster with Autoscaling on AWS

November 13, 2019

The autoscaling groups created by Kops is only to keep a number of nodes up without any scaling policy; The idea of the following article is create a dynamic Kubernetes cluster, measure the load of the nodes and scale up or down. To do that we need to set up some tags to the worker nodes, give permission to the worker nodes to reach the AWS service autoscaling and deploy the tool Kubernetes autoscaler.

The cluster will be scale when:

  • There are pods that failed to run in the cluster due to insufficient resources.
  • There are nodes in the cluster that have been underutilized for an extended period of time and their pods can be placed on other existing nodes.

The environment:

The version of "Kubernetes autoscaler" need to be the same as your Kubernetes cluster version.

Read more

Airflow: Kubernetes Operator

November 4, 2019

DAG example using KubernetesPodOperator, the idea is run a Docker container in Kubernetes from Airflow every 30 minutes.

Features:

  • Scheduled every 30 minutes.
  • Set environment variable for the pod RULES.
  • Run the pods in the namespace default.
  • Mount a volume to the container. It's just an example mounting the /tmp from host.
  • Set imagePullSecrets for private Docker registry.
Read more