Is there an official limit to nameserver indirection?

Glue records are typically unavailable if a domain and its nameserver don't share a TLD, and technically aren't required if they don't share the same second-level domain, which could lead to extra steps to resolve a domain. The resolver must first look up the address of the nameserver before it can find the address for your domain. But theoretically you could add more steps in there than just those two.

The question here is, how long is that chain allowed to be?

if xyz.com uses nameserver ns1.xyz.info,
and xyz.info uses nameserver ns1.xyz.co,
and xyz.co uses nameserver ns1.xyz.cc,
and xyz.cc uses nameserver ns1.xyz.co.uk, ... and so forth

... you could end up with a very long chain for the resolver to untangle before it can resolve the name you originally wanted.

Presumably there's a practical limit -- BIND should only be willing to traverse so many links, otherwise there's the potential for a denial-of-service. But is there an official limit? Some number of steps beyond which the resolver is officially not required to proceed?

if xyz.com uses nameserver ns1.xyz.info,

In this case your local resolver will first ask the servers for .com (eg a.gtld-servers.net) where to find the name servers for the xyz.com domain. The .com domain server will usually provide glue records for the IP addresses of the name servers for the xyz.com domain.

eg:

$ dig  gmail.com @a.gtld-servers.net 

; <<>> DiG 9.9.3-rpz2+rl.13214.22-P2-Ubuntu-1:9.9.3.dfsg.P2-4ubuntu1.1 <<>> gmail.com @a.gtld-servers.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46893
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 5
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;gmail.com.         IN  A

;; AUTHORITY SECTION:
gmail.com.      172800  IN  NS  ns2.google.com.
gmail.com.      172800  IN  NS  ns1.google.com.
gmail.com.      172800  IN  NS  ns3.google.com.
gmail.com.      172800  IN  NS  ns4.google.com.

;; ADDITIONAL SECTION:
ns2.google.com.     172800  IN  A   216.239.34.10
ns1.google.com.     172800  IN  A   216.239.32.10
ns3.google.com.     172800  IN  A   216.239.36.10
ns4.google.com.     172800  IN  A   216.239.38.10

;; Query time: 375 msec
;; SERVER: 192.5.6.30#53(192.5.6.30)
;; WHEN: Thu Jul 10 01:10:57 EST 2014
;; MSG SIZE  rcvd: 181

In my experience your statement that "Glue records are typically unavailable if a domain and its nameserver don't share a TLD" is simply not true. It is however dependent on the data provided by the registrar for the domain, and their policies vary. Some require the IP to be specified. Some leave it up to the domain owner. I think I remember one in Australia that doesn't support them. If the number of registrars dealing with your country's domain is small, then perhaps this might be true within that part of the domain space, but for the net as a whole this is atypical.

It's certainly good practice for domain owners to provide Glue records, but sometimes the ability to specify a DNS server by name without nailing down the IP is seen as providing flexibility, and many domain owners don't understand the performance issue this creates.

If you're providing a DNS reporting tool, is it really the absolute limit on such misconfiguration that you are interested in? Surely it's more to the point for you to report the missing glue records as a warning if even one such missing glue record issue presents. You probably want to track at least a few indirections such as you provide, (and report theit missing glue records) but there must be some limits on how far you should follow this. I'd be quite happy if a DNS reporting tool warned of the first 3 or so, as I'd only really be interested in either adding the glue records to my domain, or switching DNS provider for the domain to one that's more competent.

I'm doubtful of your proposed DOS approach on BIND, as BIND will cache the info it collects about the location of name servers. An attacker would have to set up a lot of domains without glue and then make a lot of queries about them. The cost of setting up domains which would likely be cancelled by the registrar after use is likely to make this unappealing to an attacker.