Received fatal alert: handshake_failure through SSLHandshakeException

I have a problem with authorized SSL connection. I have created Struts Action that connects to external server with Client Authorized SSL certificate. In my Action I am trying to send some data to bank server but without any luck, because I have as a result from server the following error:

error: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

My Method from my Action class that sends data to server

//Getting external IP from host
    URL whatismyip = new URL("http://automation.whatismyip.com/n09230945.asp");
    BufferedReader inIP = new BufferedReader(new InputStreamReader(whatismyip.openStream()));

    String IPStr = inIP.readLine(); //IP as a String

    Merchant merchant;

    System.out.println("amount: " + amount + ", currency: " + currency + ", clientIp: " + IPStr + ", description: " + description);

    try {

        merchant = new Merchant(context.getRealPath("/") + "merchant.properties");

    } catch (ConfigurationException e) {

        Logger.getLogger(HomeAction.class.getName()).log(Level.INFO, "message", e);
        System.err.println("error: " + e.getMessage());
        return ERROR;
    }

    String result = merchant.sendTransData(amount, currency, IPStr, description);

    System.out.println("result: " + result);

    return SUCCESS;

My merchant.properties file:

bank.server.url=https://-servernameandport-/
https.cipher=-cipher-

keystore.file=-key-.jks
keystore.type=JKS
keystore.password=-password-
ecomm.server.version=2.0

encoding.source=UTF-8
encoding.native=UTF-8

For the first time I thought this is a certificate problem, I converted it from .pfx to .jks, but I have the same error, with no changes.

Solution 1:

The handshake failure could have occurred due to various reasons:

• Incompatible cipher suites in use by the client and the server. This would require the client to use (or enable) a cipher suite that is supported by the server.
• Incompatible versions of SSL in use (the server might accept only TLS v1, while the client is capable of only using SSL v3). Again, the client might have to ensure that it uses a compatible version of the SSL/TLS protocol.
• Incomplete trust path for the server certificate; the server's certificate is probably not trusted by the client. This would usually result in a more verbose error, but it is quite possible. Usually the fix is to import the server's CA certificate into the client's trust store.
• The cerificate is issued for a different domain. Again, this would have resulted in a more verbose message, but I'll state the fix here in case this is the cause. The resolution in this case would be get the server (it does not appear to be yours) to use the correct certificate.

Since, the underlying failure cannot be pinpointed, it is better to switch on the -Djavax.net.debug=all flag to enable debugging of the SSL connection established. With the debug switched on, you can pinpoint what activity in the handshake has failed.

Update

Based on the details now available, it appears that the problem is due to an incomplete certificate trust path between the certificate issued to the server, and a root CA. In most cases, this is because the root CA's certificate is absent in the trust store, leading to the situation where a certificate trust path cannot exist; the certificate is essentially untrusted by the client. Browsers can present a warning so that users may ignore this, but the same is not the case for SSL clients (like the HttpsURLConnection class, or any HTTP Client library like Apache HttpComponents Client).

Most these client classes/libraries would rely on the trust store used by the JVM for certificate validation. In most cases, this will be the cacerts file in the JRE_HOME/lib/security directory. If the location of the trust store has been specified using the JVM system property javax.net.ssl.trustStore, then the store in that path is usually the one used by the client library. If you are in doubt, take a look at your Merchant class, and figure out the class/library it is using to make the connection.

Adding the server's certificate issuing CA to this trust store ought to resolve the problem. You can refer to my answer on a related question on getting tools for this purpose, but the Java keytool utility is sufficient for this purpose.

Warning: The trust store is essentially the list of all CAs that you trust. If you put in an certificate that does not belong to a CA that you do not trust, then SSL/TLS connections to sites having certificates issued by that entity can be decrypted if the private key is available.

Update #2: Understanding the output of the JSSE trace

The keystore and the truststores used by the JVM are usually listed at the very beginning, somewhat like the following:

keyStore is : 
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
trustStore is: C:\Java\jdk1.6.0_21\jre\lib\security\cacerts
trustStore type is : jks
trustStore provider is : 

If the wrong truststore is used, then you'll need to re-import the server's certificate to the right one, or reconfigure the server to use the one listed (not recommended if you have multiple JVMs, and all of them are used for different needs).

If you want to verify if the list of trust certs contains the required certs, then there is a section for the same, that starts as:

adding as trusted cert:
  Subject: CN=blah, O=blah, C=blah
  Issuer:  CN=biggerblah, O=biggerblah, C=biggerblah
  Algorithm: RSA; Serial number: yadda
  Valid from SomeDate until SomeDate

You'll need to look for if the server's CA is a subject.

The handshake process will have a few salient entries (you'll need to know SSL to understand them in detail, but for the purpose of debugging the current problem, it will suffice to know that a handshake_failure is usually reported in the ServerHello.

1. ClientHello

A series of entries will be reported when the connection is being initialized. The first message sent by the client in a SSL/TLS connection setup is the ClientHello message, usually reported in the logs as:

*** ClientHello, TLSv1
RandomCookie:  GMT: 1291302508 bytes = { some byte array }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA]
Compression Methods:  { 0 }
***

Note the cipher suites used. This might have to agree with the entry in your merchant.properties file, for the same convention might be employed by the bank's library. If the convention used is different, there is no cause of worry, for the ServerHello will state so, if the cipher suite is incompatible.

2. ServerHello

The server responds with a ServerHello, that will indicate if the connection setup can proceed. Entries in the logs are usually of the following type:

*** ServerHello, TLSv1
RandomCookie:  GMT: 1291302499 bytes = { some byte array}
Cipher Suite: SSL_RSA_WITH_RC4_128_SHA
Compression Method: 0
***

Note the cipher suite that it has chosen; this is best suite available to both the server and the client. Usually the cipher suite is not specified if there is an error. The certificate of the server (and optionally the entire chain) is sent by the server, and would be found in the entries as:

*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=server, O=server's org, L=server's location, ST =Server's state, C=Server's country
  Signature Algorithm: SHA1withRSA, OID = some identifer

.... the rest of the certificate
***

If the verification of the certificate has succeeded, you'll find an entry similar to:

Found trusted certificate:
[
[
  Version: V1
  Subject: OU=Server's CA, O="Server's CA's company name", C=CA's country
  Signature Algorithm: SHA1withRSA, OID = some identifier

One of the above steps would not have succeeded, resulting in the handshake_failure, for the handshake is typically complete at this stage (not really, but the subsequent stages of the handshake typically do not cause a handshake failure). You'll need to figure out which step has failed, and post the appropriate message as an update to the question (unless you've already understood the message, and you know what to do to resolve it).

Solution 2:

The handshake failure could be a buggy TLSv1 protocol implementation.

In our case this helped with java 7:

java -Dhttps.protocols=TLSv1.2,TLSv1.1,TLSv1 

The jvm will negotiate in this order. The servers with the latest update will do 1.2, the buggy ones will go down to v1 and that works with the similar v1 in java 7.

Solution 3:

Installing Java Cryptography Extension (JCE) Unlimited Strength (for JDK7 | for JDK8) might fix this bug. Unzip the file and follow the readme to install it.

Solution 4:

This can also happend when the client needs to present a certificate. After the server lists the certificate chain, the following can happen:

3. Certificate Request The server will issue a certificate request from the client. The request will list all of the certificates the server accepts.

*** CertificateRequest
Cert Types: RSA
Cert Authorities:
<CN=blah, OU=blah, O=blah, L=blah, ST=blah, C=blah>
<CN=yadda, DC=yadda, DC=yadda>
<CN=moreblah, OU=moreblah, O=moreblah, C=moreblah>
<CN=moreyada, OU=moreyada, O=moreyada, C=moreyada>
... the rest of the request
*** ServerHelloDone

4. Client Certificate Chain This is the certificate the client is sending to the server.

*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: EMAILADDRESS=client's email, CN=client, OU=client's ou, O=client's Org, L=client's location, ST=client's state, C=client's Country
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5
  ... the rest of the certificate
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1    
... key exchange info 

If there isn't a certificate in the chain and the server requires one, you'll get the handshake error here. A likely cause is the path to your certificate wasn't found.

5. Certificate Verify The client asks the server to verify the certificate

*** CertificateVerify
... payload of verify check

This step will only happen if you are sending a certificate.

6. Finished The server will respond with a verify response

*** Finished
verify_data:  { 345, ... }

Solution 5:

I don't think this solves the problem to the first questioner, but for googlers coming here for answers:

On update 51, java 1.8 prohibited[1] RC4 ciphers by default, as we can see on the Release Notes page:

Bug Fix: Prohibit RC4 cipher suites

RC4 is now considered as a compromised cipher.

RC4 cipher suites have been removed from both client and server default enabled cipher suite list in Oracle JSSE implementation. These cipher suites can still be enabled by SSLEngine.setEnabledCipherSuites() and SSLSocket.setEnabledCipherSuites() methods. See JDK-8077109 (not public).

If your server has a strong preference for this cipher (or use only this cipher) this can trigger a handshake_failure on java.

You can test connecting to the server enabling RC4 ciphers (first, try without enabled argument to see if triggers a handshake_failure, then set enabled:

import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import java.io.*;

import java.util.Arrays;

/** Establish a SSL connection to a host and port, writes a byte and
 * prints the response. See
 * http://confluence.atlassian.com/display/JIRA/Connecting+to+SSL+services
 */
public class SSLRC4Poke {
    public static void main(String[] args) {
        String[] cyphers;
        if (args.length < 2) {
            System.out.println("Usage: "+SSLRC4Poke.class.getName()+" <host> <port> enable");
            System.exit(1);
        }
        try {
            SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();
            SSLSocket sslsocket = (SSLSocket) sslsocketfactory.createSocket(args[0], Integer.parseInt(args[1]));
        
            cyphers = sslsocketfactory.getSupportedCipherSuites();
            if (args.length ==3){
                sslsocket.setEnabledCipherSuites(new String[]{
                    "SSL_DH_anon_EXPORT_WITH_RC4_40_MD5",
                    "SSL_DH_anon_WITH_RC4_128_MD5",
                    "SSL_RSA_EXPORT_WITH_RC4_40_MD5",
                    "SSL_RSA_WITH_RC4_128_MD5",
                    "SSL_RSA_WITH_RC4_128_SHA",
                    "TLS_ECDHE_ECDSA_WITH_RC4_128_SHA",
                    "TLS_ECDHE_RSA_WITH_RC4_128_SHA",
                    "TLS_ECDH_ECDSA_WITH_RC4_128_SHA",
                    "TLS_ECDH_RSA_WITH_RC4_128_SHA",
                    "TLS_ECDH_anon_WITH_RC4_128_SHA",
                    "TLS_KRB5_EXPORT_WITH_RC4_40_MD5",
                    "TLS_KRB5_EXPORT_WITH_RC4_40_SHA",
                    "TLS_KRB5_WITH_RC4_128_MD5",
                    "TLS_KRB5_WITH_RC4_128_SHA"
                });     
            }

            InputStream in = sslsocket.getInputStream();
            OutputStream out = sslsocket.getOutputStream();

            // Write a test byte to get a reaction :)
            out.write(1);

            while (in.available() > 0) {
                System.out.print(in.read());
            }
            System.out.println("Successfully connected");

        } catch (Exception exception) {
            exception.printStackTrace();
        }
    }
}

1 - https://www.java.com/en/download/faq/release_changes.xml