BIND9 - dig is unable to resolve from different server

$ named -v
BIND 9.16.1-Ubuntu (Stable Release) <id:d497c32>

I have 3 servers configured in digitalocean nyc1 all in the same subnet

on server01 - I have installed bind9 and configured the zones and it works great

server01 $ dig @10.116.16.2 -p 53 ns1.prod.nyc1.example

...
;; ANSWER SECTION:
ns1.prod.nyc1.example. 43200    IN  A   10.116.16.2

This works well when I am on server01

from server02 (which is also in the same subnet)

server02 $ dig @10.116.16.2 -p 53 ns1.prod.nyc1.example

; <<>> DiG 9.16.1-Ubuntu <<>> @10.116.16.2 -p 53 ns1.prod.nyc1.example
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

However I can telnet to it from server02

server02 $ telnet 10.116.16.2 53
Trying 10.116.16.2...
Connected to 10.116.16.2.
Escape character is '^]'.

and now when I restart sudo systemctl restart bind9 on server01, it disconnects on server02

Here is the /etc/bind/named.conf.options

options {
        directory "/var/cache/bind";
        recursion yes;
        listen-on port 53 { any; };
        allow-query { any; };
        allow-recursion { any; };


        dnssec-enable no;
        dnssec-validation no;

        auth-nxdomain no;    # conform to RFC1035
};

include "/etc/bind/consul.conf";

What am I doing wrong? lookup works on server01 but from a different server it does not work

I looked at - Internal DNS Setup [Bind9] , unable to dig from another machine, but able to dig locally but that does not solve my problem

Solution 1:

telnet uses TCP while DNS (dig) uses UDP by default but also TCP, and this difference can explain what you observe.

Try dig +tcp to force a TCP connection, and it will probably succeed, proving that you are filtering UDP somewhere where you shouldn't.

Remove this filtering on UDP in your systems and everything should start to work as expected.