Schedule pods to start and stop in Kubernetes by date and time

June 24, 2021 - Last updated: July 12, 2021

Kubernetes by default doesn't provide this feature but you can use Another Autoscaler controller for that.

This approach helps us to save some money in the Kubernetes clusters with GPU instances (P2 and G4DN); Developers using these types of instances generally use them during working hours, so we can terminate pods after working hours and our Cluster Autoscaler will scale-down instances that are idle.

Another Autoscaler page: https://github.com/dignajar/another-autoscaler

How to install Another Autoscaler

You can use the following K8s manifest, or download it and change it as you need.

kubectl apply -f https://raw.githubusercontent.com/dignajar/another-autoscaler/master/kubernetes/full.yaml

The manifest will install Another Autoscaler in the "default" namespace.

How Another Autoscaler works

Another Autoscaler reads the annotations inside the deployments and perform different action; The annotations prefix is another-autoscaler.io.

  • Another Autoscaler use Cron syntaxt for the date and time.
  • The date and time have to be in UTC.

For example:

# Scale down the replicas to zero at 18:00 hs every day.
another-autoscaler.io/stop-time: "00 18 * * *"

# Scale up the replicas to one at 20:00 hs every day.
another-autoscaler.io/start-time: "00 20 * * *"

How to use Another Autoscaler

Add to your deployment manifest the annotations, for example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
  annotations:
    another-autoscaler.io/start-time: 00 14 * * *
    another-autoscaler.io/stop-time: 00 15 * * *
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: 'nginx:latest'
          ports:
            - containerPort: 80

Alternative you can use kubectl to apply the annotations on the fly to a deployment.

kubectl annotate deploy nginx another-autoscaler.io/start-time="00 14 * * *"
kubectl annotate deploy nginx another-autoscaler.io/stop-time="00 15 * * *"

Limitations / Issues / GitOps

  • If you are using GitOps (Flux, ArgoCD, etc) to apply changes into your Kubernetes cluster, you have to remove the field replica from the deployment manifest to avoid overwriting between the GitOps tools and the Another Autoscaler controller.

Related posts