Helm install spark-operator tries to download a version that does not exist and cannot be force to use the correct one

I am trying to install ANY working version of spark-operator https://console.cloud.google.com/gcr/images/spark-operator/GLOBAL/spark-operator?tag=v1beta2-1.3.1-3.1.1 on my local kubernetes. However, spark pod is stuck on ImagePullBackOff trying to download a version that does not exist.

Commands:

helm repo add spark-operator https://googlecloudplatform.github.io/spark-on-k8s-operator

helm install v1beta2-1.2.1-3.0.0 spark-operator/spark-operator --namespace spark-operator6 --create-namespace

kubectl get pods -n spark-operator6

NAME READY STATUS RESTARTS AGE v1beta2-1.2.1-3.0.0-spark-operator-67577fd4d4-m9zmw 0/1 ImagePullBackOff 0 6s

kubectl describe pod v1beta2-1.2.1-3.0.0-spark-operator-67577fd4d4-m9zmw

Image: gcr.io/spark-operator/spark-operator:v1beta2-1.3.1-3.1.1 - a different one! Failed to pull image "gcr.io/spark-operator/spark-operator:v1beta2-1.3.1-3.1.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/spark-operator/spark-operator:v1beta2-1.3.1-3.1.1 not found: manifest unknown: Failed to fetch "v1beta2-1.3.1-3.1.1" from request "/v2/spark-operator/spark-operator/manifests/v1beta2-1.3.1-3.1.1"

Now this just seems like an incorrect version. Tried to override it via helm install flag --operatorVersion, but that is completely ignored.

Took a look at its template via helm template v1beta2-1.2.1-3.0.0 spark-operator/spark-operator --namespace spark-operator6 --set operatorVersion=v1beta2-1.2.1-3.0.0 > template.yaml and all the app.kubernetes.io/instance are on correct version, but ALL app.kubernetes.io/version were showing the non-existing v1beta2-1.3.1-3.1.1. Corrected it and tried to install with the correct values via

helm install -f template.yaml v1beta2-1.2.1-3.0.0 spark-operator/spark-operator --namespace spark-operator6

but this was completely ignored again.

I feel like I am missing something very basic here, helm install cannot possibly be that broken. Any help is much appreciated, thanks.

Solution 1:

Try this one

helm upgrade -i my-release spark-operator/spark-operator --namespace spark-operator --set image.tag=v1beta2-1.3.2-3.1.1 --set image.repository=ghcr.io/googlecloudplatform/spark-operator

the key thing here is the image.tag and image.repository.

I am assuming you have a namespace for spark-operator. Please adjust namespace according to your requirements.