Kubernetes: Delete pods older than X days

February 14, 2020

I'm working with Airflow and the Kubernetes operator which deploys from Airflow a container to Kubernetes. The pods created by Airflow have ended so the final status of the pod could be Succeeded or Failed. I was looking for to delete the pods created by Airflow but older than 5 days, so we can still read the logs from the pods until they are deleted.

I created a Python script that uses the Kubernetes API and a service account with permissions for reading and delete pods from a particular namespace. Also, I create a Docker image base on Python to execute the script and finally create a cronjob in Kubernetes to trigger this Docker every day at 2 am.

I don't want to copy and paste my README.md from my Github repository so here is the link:

Here is the link to the Docker image in Docker hub.


Airflow: Kubernetes Operator

November 4, 2019

DAG example using KubernetesPodOperator, the idea is run a Docker container in Kubernetes from Airflow every 30 minutes.

Features:

  • Scheduled every 30 minutes.
  • Set environment variable for the pod RULES.
  • Run the pods in the namespace default.
  • Mount a volume to the container. It's just an example mounting the /tmp from host.
  • Set imagePullSecrets for private Docker registry.
Read more