Debugging client certificate issues on IIS

Solution 1:

We finally found the answer to this: at fault is KB931125. A blog article describes the KB as only intended for client targets, but got shipped to servers which results in the CA list being sent by IIS to be truncated. It doesn't really explain why we couldn't see the transmitted CA list with OpenSSL, but ultimately it did get us to the root cause.

The blog article points to a further MSDN article that describes the issue in more detail:

These problems may occur if a TLS/SSL server contains many entries in the trusted root certification list. The server sends a list of trusted certificate authorities to the client if the following conditions are true:

• The server uses the Transport Layer Security (TLS)/SSL protocol to encrypt network traffic.
• Client certificates are required for authentication during the authentication handshake process.

This list of trusted certificate authorities represents the authorities from which the server can accept a client certificate. To be authenticated by the server, the client must have a certificate that is present in the chain of certificates to a root certificate from the server's list. This is because the client certificate is always the end-entity certificate at the end of the chain. The client certificate isn't part of the chain.

Currently, the maximum size of the trusted certificate authorities list that the Schannel security package supports is 16 KB in Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012.

An event log entry gets logged as well that says (which we missed the first time we looked at this):

When asking for client authentication, this server sends a list of trusted certificate authorities to the client. The client uses this list to choose a client certificate that is trusted by the server. Currently, this server trusts so many certificate authorities that the list has grown too long. This list has thus been truncated. The administrator of this machine should review the certificate authorities trusted for client authentication and remove those that do not really need to be trusted.

The MSDN article does list a fix:

Delete the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\Certificates

To do this, follow these steps:

1. Start Registry Editor
2. Locate the following registry subkey: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\AuthRoot
3. Right-click and then delete the key that is called "Certificates"