Airflow: Kubernetes Operator

November 4, 2019

DAG example using KubernetesPodOperator, the idea is run a Docker container in Kubernetes from Airflow every 30 minutes.

Features:

  • Scheduled every 30 minutes.
  • Set environment variable for the pod RULES.
  • Run the pods in the namespace default.
  • Mount a volume to the container. It's just an example mounting the /tmp from host.
  • Set imagePullSecrets for private Docker registry.
Read more


Kubectl autocompletion in MacOS

March 20, 2019

MacOS comes with a really old bash version. Check the version from the environment variable..

$ echo $BASH_VERSION
3.2.57(1)-release

We can install a newer version of bash via Homebrew package manager, get Homebrew from here https://brew.sh.

Read more