Kubernetes by default doesn't provide this feature but you can use Another Autoscaler controller for that.
We are using this tool in my company to save some money on the Kubernetes clusters with GPU instances (P2 and G4DN); Developers using these types of instances generally use them during working hours, so we can terminate pods after working hours and our Cluster Autoscaler will scale-down instances that are idle.
Another Autoscaler page: https://github.com/dignajar/another-autoscaler
How to install Another Autoscaler
You can use the following K8s manifest, or download it and change it as you need.
$ curl https://raw.githubusercontent.com/dignajar/another-autoscaler/master/kubernetes/full.yaml -o /tmp/another.yaml
$ vi /tmp/another.yaml
$ kubectl -f /tmp/another.yaml
$ kubectl get pods -n another
The manifest will create the namespace "another" and install Another Autoscaler on it.
How Another Autoscaler works
Another Autoscaler reads the annotations inside the deployments and perform different action; The annotations prefix is another-autoscaler.io
.
- Another Autoscaler use Cron syntaxt for the date and time.
- The date and time have to be in UTC.
For example:
# Scale down the replicas to zero at 18:00 hs every day.
another-autoscaler.io/stop-time: "00 18 * * *"
# Scale up the replicas to one at 20:00 hs every day.
another-autoscaler.io/start-time: "00 20 * * *"
How to use Another Autoscaler
Add to your deployment manifest the annotations, for example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
annotations:
another-autoscaler.io/start-time: 00 14 * * *
another-autoscaler.io/stop-time: 00 15 * * *
spec:
replicas: 0
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 'nginx:latest'
ports:
- containerPort: 80
Alternative you can use kubectl to apply the annotations on the fly to a deployment.
kubectl annotate deploy nginx another-autoscaler.io/start-time="00 14 * * *"
kubectl annotate deploy nginx another-autoscaler.io/stop-time="00 15 * * *"
Limitations / Issues / GitOps
- If you are using GitOps (Flux, ArgoCD, etc) to apply changes into your Kubernetes cluster, you have to remove the field
replica
from the deployment manifest to avoid overwriting between the GitOps tools and the Another Autoscaler controller.