cluster-autoscaler deployment fails with "1 Too many pods, 3 node(s) didn't match Pod's node affinity/selector"

Solution 1:

Posting the answer out of comments.

There are podAffinity rules in place so first thing to check is if any errors in scheduling are presented. Which is the case:

0/4 nodes are available: 1 Too many pods, 3 node(s) didn't match Pod's node affinity/selector.

Since there are 1 control plane (on which pod is supposed to be scheduled) and 3 worked nodes, that leads to the error 1 Too many pods related to the control plane.

Since cluster is running in AWS, there's a known limitation about amount of network interfaces and private IP addresses per machine type - IP addresses per network interface per instance type.

t3.small was used which has 3 interfaces and 4 IPs per interface = 12 in total which was not enough.

Scaling up to t3.medium resolved the issue.

Credits to Jonas's answer about the root cause.