Nginx Ingress Controller - Failed Calling Webhook

I set up a k8s cluster using kubeadm (v1.18) on an Ubuntu virtual machine. Now I need to add an Ingress Controller. I decided for nginx (but I'm open for other solutions). I installed it according to the docs, section "bare-metal":

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-0.31.1/deploy/static/provider/baremetal/deploy.yaml

The installation seems fine to me:

kubectl get all -n ingress-nginx

NAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-b8smg        0/1     Completed   0          8m21s
pod/ingress-nginx-admission-patch-6nbjb         0/1     Completed   1          8m21s
pod/ingress-nginx-controller-78f6c57f64-m89n8   1/1     Running     0          8m31s

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/ingress-nginx-controller             NodePort    10.107.152.204   <none>        80:32367/TCP,443:31480/TCP   8m31s
service/ingress-nginx-controller-admission   ClusterIP   10.110.191.169   <none>        443/TCP                      8m31s

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           8m31s

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-78f6c57f64   1         1         1       8m31s

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/ingress-nginx-admission-create   1/1           2s         8m31s
job.batch/ingress-nginx-admission-patch    1/1           3s         8m31s

However, when trying to apply a custom Ingress, I get the following error:

Error from server (InternalError): error when creating "yaml/xxx/xxx-ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post https://ingress-nginx-controller-admission.ingress-nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s: Temporary Redirect

Any idea what could be wrong?

I suspected DNS, but other NodePort services are working as expected and DNS works within the cluster.

The only thing I can see is that I don't have a default-http-backend which is mentioned in the docs here. However, this seems normal in my case, according to this thread.

Last but not least, I tried as well the installation with manifests (after removing ingress-nginx namespace from previous installation) and the installation via Helm chart. It has the same result.

I'm pretty much a beginner on k8s and this is my playground-cluster. So I'm open to alternative solutions as well, as long as I don't need to set up the whole cluster from scratch.

Update: With "applying custom Ingress", I mean: kubectl apply -f <myIngress.yaml>

Content of myIngress.yaml

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /someroute/fittingmyneeds
        pathType: Prefix
        backend:
          serviceName: some-service
          servicePort: 5000

Solution 1:

Another option you have is to remove the Validating Webhook entirely:

kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

I found I had to do that on another issue, but the workaround/solution works here as well.

This isn't the best answer; the best answer is to figure out why this doesn't work. But at some point, you live with workarounds.

I'm installing on Docker for Mac, so I used the cloud rather than baremetal version:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/cloud/deploy.yaml

Solution 2:

In my case I'd mixed the installations up. I resolved the issue by executing the following steps:

$ kubectl get validatingwebhookconfigurations 

I iterated through the list of configurations received from the above steps and deleted the configuration using

$ `kubectl delete validatingwebhookconfigurations [configuration-name]`

Solution 3:

I've solved this issue. The problem was that you use Kubernetes version 1.18, but the ValidatingWebhookConfiguration in current ingress-Nginx uses the oldest API; see the doc: https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites

Ensure that the Kubernetes cluster is at least as new as v1.16 (to use admissionregistration.k8s.io/v1), or v1.9 (to use admissionregistration.k8s.io/v1beta1).

And in current yaml :

 # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
    # before changing this value, check the required kubernetes version
    # https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
apiVersion: admissionregistration.k8s.io/v1beta1

and in rules :

apiVersions:
          - v1beta1

So you need to change it on v1 :

apiVersion: admissionregistration.k8s.io/v1

and add rule -v1 :

apiVersions:
          - v1beta1
          - v1

After you change it and redeploy -your custom ingress service will deploy sucessfull