Kubernetes Cluster Autoscaling – A Refresher

A topic that frequently comes up these days is Container Autoscaling. As with many concepts in technology, it is an overloaded term. There are not one but three kinds of auto-scaling mechanisms in Kubernetes. 

These are – 

1. Horizontal Pod Autoscaling (hpa)
2. Vertical Pod Autoscaling (vpa)
3. Cluster Autoscaling

The first two operate autoscalers at the Application/CaaS/PaaS layer and the third at the IaaS layer.

The Cluster Autoscaler

The Cluster Autoscaler is the first and most commonly used tool. It can also be the easiest to setup. It is basically a tool that maintains the state of the K8s cluster by adding or removing nodes based on specific metrics. It adds nodes when it detects a given number of pending pods due to node shortages. It removes nodes when metric utilization in nodes falls below a certain level. 

Illustration – Cluster Autoscaler Workflow

The workflow in the Cluster Autoscaling implementation of AWS EKS (based on K8s) is shown above. It can be visualized that the Cluster Autoscaler ensures the correct amount of node utilization without having to over provision node capacity. In the cloud, it translates to much better usage of the underlying node pools.  Having said this, there are scenarios where cluster autoscaler will not function.

These include 

• Pod disruption budget (PDB) controls the number of pods of a replicated application that must be present. This is for applications such as databases which always need a quorum to maintain a running application.The Cluster Autoscaler will not take down pods that have low PodDisruptionBudget
• Pods that use local storage
• Kube-system pods that don’t run on the node by default or have a low PDB
• Pods that are not created by a deployment, replica set, stateful set etc i.e not backed by a controller object 
• Pods with local storage
• Pods that cannot be moved elsewhere due to various constraints (lack of resources, non-matching node selectors or affinity, matching anti-affinity, etc)

Key Considerations in Using Cluster Autoscaler 

1. To use the cluster autoscaler correctly, the different nodes that make up your cluster & run application pods to uniformly be of the same instance type/capacity/labels/taints etc. The reason for this is that the cluster autoscaler assumes that nodes are the same capacity and uses derived templates to schedule pods. It is also important to note that cloud providers such as AWS enable cluster autoscaler to work with a range of their cloud services such as a range of EC2 instances, EBS volumes and GPU accelerators. 
2. Pods should specify their resource requests
3. Leverage PDBs for both kube-system pods and application pods as well to ensure that pods of sensitive applications are not deleted abruptly 
4. When using a cloud provider such as AWS, leverage features such as Managed node groups (MNGs). MNGs are equipped with advanced management features for Cluster Autoscaler such as automatic Amazon EC2 Auto Scaling group discovery and graceful node termination.
5. For large scale applications, node groups should be scoped across AZs 
6. To keep costs down, the cluster autpscaler can scale node groups right up from zero

Conclusion

Cluster autoscaling is typically the easiest autoscaler to configure and to use. However there may be situations where it may not be the best option for an application. If the application is not highly critical or if a workload dont change or spike that often, manual scaling may well be the way to go.