Kubernetes Not Scheduling CronJob
A couple of other things you can check:
- Do you have any cron pods with a "failed" status? If you do, check those pods for why.
- Did it used to work and then suddenly stop?
- Does the cronjob resource have anything in the events?
kubectl describe cronjob health-status-cron -n tango
- Does the code your cron runs take > 1 minute to complete? If so, your schedule is too aggressive, and you might want to loosen the schedule
- The cronjob controller also has some limitations you may want to check: https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#cron-job-limitations. Specifically the concept of "missed jobs". If the cronjob controller "misses" scheduling 100 or more jobs, it will "freeze" the job and not schedule it anymore. Do you scale down the cluster or similar when it is not in use?
- Do you have any custom/third-party webhooks or plugins installed in the cluster? These can interfere with pod creation.
- Do you have any jobs created in the namespace?
kubectl get jobs -n tango
If you find a ton of job objects, check them to see why they did not generate pods.
I encountered a somewhat similar issue in 2020 (writeup has a link to the issue I raised in the Kubernetes project itself): https://blenderfox.com/2020/08/07/the-snowball-effect-in-kubernetes/