IIS Windows Authentication with DNS

I asked this question on Stackoverflow already, but I realized this might be a better place to post it.

I am using IIS 10 on Windows Server 2016.

I have successfully configured IIS to use Windows Authentication when going onto the site. This works well when using the IP address or hostname of the server.

However I am going to host several sites on the same server so I tried configuring some DNS records to point to the IP of the server (A records) and by defining a hostname in the IIS binding for that site I have managed to properly connect to the right site when using that name in the browser.

But for some reason, this breaks the Windows Authentication. When putting the username and password into the browser login-box it simply fails to authenticate.

I read this answer but it didn't provide a solution (https://stackoverflow.com/questions/15366649/iis-windows-authentication-weirdness)

I tried writing the username in different ways:

username
domain\username
domain.local\username

but it doesn't make any difference.

I looked at this page but it doesn't seem to be applicable.

I was advised to try turning off reverse DNS checking in the IIS metbase with this command: adsutil set w3svc/EnableReverseDNS FALSE. In order to do it, I had to add the role IIS 6 Management Compatibility.

Does anyone know how to make this work?

I've assumed it's a domain member due to the username syntax you posted.

If you've used A records, for Kerberos authentication to succeed, you need a Service Principal Name defined for each name, owned by the process identity of the target site. (If you've left it at the default ApplicationPoolIdentity, LocalService, NetworkService or Local System, that's going to be the machine account.)

There's a side effect which you can exploit to test the theory, though, which is that IE does CNAME to A Record resolution and queries the SPN of the resolved target name...

So if you had fakehost.mydomain as a CNAME for MACHINENAME, IE would resolve MACHINENAME and then use HTTP/MACHINENAME as the SPN it was looking up - meaning you wouldn't have to create any additional SPNs. This works as long as the identity for all sites is the machine.

Now, the odd thing here is that Kerb authentication shouldn't be the only option, and I'm interested that the NTLM fallback didn't work at all when Kerb failed to retrieve a ticket, so there might be some other settings or complexity here complicating things. That's where Extended Protection might be problematic, at a guess. Or perhaps the auth settings are no longer Negotiate (which means Kerb and NTLM if Kerb fails) as part of your troubleshooting/setup.

Finally, the wonderful thing about dotted host names is that they tend to invoke a proxy - if you've a proxy defined in IE, disable it for your testing.

Super-finally, it's very unlikely to be anything related to IIS's view of DNS, so you can uninstall the IIS Management 6 stuff - nothing you can configure or break in IIS requires the use of the legacy compatibility stuff to fix it.