Kubernetes by default doesn't provide this feature but you can use Another Autoscaler controller for that.

We are using this tool in my company to save some money on the Kubernetes clusters with GPU instances (P2 and G4DN); Developers using these types of instances generally use them during working hours, so we can terminate pods after working hours and our Cluster Autoscaler will scale-down instances that are idle.

Another Autoscaler page: https://github.com/dignajar/another-autoscaler

Time for GPU and Kubernetes, the idea is to provide an instance group with Kubernetes' nodes with GPU, also to save some money we can start the autoscaling group from 0 instances.

The environment

• AWS
• G4DN instances also tested on P2 instances
• Kubernetes 1.18 deployed via Kops
• Cluster Autoscaler 1.18
• GPU Operator

Assuming you have deployed AWS Load Balancer Controller, the following steps are how to configure one ALB to expose all your services, also services cross namespaces.

Default configuration for the ALB "dev" with the following features:

• HTTP redirect to HTTPs.
• SSL termination, with ACM certificate provide from AWS.

The idea is to wait inside an init-container until the health check is successful. The health check, in this case, is the HTTP response code equal to 200.

• This init-container will continue after the service returns 200.
• If the init-container doesn't complete, the container will not be executed.

initContainers:
- name: wait-for-webserver
  image: curlimages/curl:latest
  command: ["/bin/sh","-c"]
  args: ["while [ $(curl -sw '%{http_code}' http://webserver.svc.cluster.local -o /dev/null) -ne 200 ]; do sleep 5; echo 'Waiting for the webserver...'; done"]

I was trying to insert a few global environment variables to every container running, so one of the options was Pod Presets but I realized this feature will be removed in Kubernetes 1.20.x, check this pull request).

One of the solution is create a ConfigMap and use the envFrom to define all of the ConfigMap's data as container environment variables.

After a wild using Kubernetes in AWS and set-up persistent volumes via EBS, I faced a problem with evicted pods after they are re-schedule to another node; The issue was the EBS volumes are dedicated by zone, makes sense because the volumes work via networking and are dedicated per datacenter, for the network latency.