KubeADM cluster: how to configure DNS properly
- doesn't allow me to access the Internet.
Your default configuration seems fine. As you've found, you don't want to change your pod resolv.conf as it needs to use the coredns service within the cluster for local dns lookups. The coredns service should be able to forward dns requests to the resolver configured in your k8s-master /etc/resolv.conf. It seems like it may be a problem with the k8s-master server internet access or with the virtualbox networking. I cloned your repo and was able to resolve internet dns from pods running on the cluster as expected.
If you take a look at the coredns configuration you will see that it's set to use /etc/resolv.conf for any unspecified domains. From k8s-master you can run kubectl -n kube-system get configmap coredns -o yaml to see the coredns config. There will be a line, forward . /etc/resolv.conf. The kubernetes docs here explain the coredns corefile configuration. You will also find that the resolve.conf in the coredns pod is the same as on the host. To verify this, you can run ps -aux | grep coredns to get the pid of one of the coredns processes. Then run nsenter -t <pid> -n cat /etc/resolv.conf and you should see that the content is the same as the host system. It should have a resolver from the virtualbox network. You can test this resolver from the host or within the coredns process to validate if it can resolve dns. You could also test against the google resolver from within the coredns process to see what you get. I would try a dns lookup as well as just a ping. This should help you narrow down what's causing your issue. Again, I tried your vagrantfile on a brand new ubuntu 20.04 server and it worked fine.
- in the search clause there are domains coming from my host machines that cause issues.
This is also the expected configuration. I wouldn't typically expect this to cause issues, but I don't know your exact configuration or what you're trying to achieve. If the solution requires that you modify the list of search domains in the pod, the place to do that would be in the pod dns policy. Hopefully solving your first issue will allow you to avoid having to customize this.