HTTPURLConnection Doesn't Follow Redirect from HTTP to HTTPS

I can't understand why Java's HttpURLConnection does not follow an HTTP redirect from an HTTP to an HTTPS URL. I use the following code to get the page at https://httpstat.us/:

import java.net.URL;
import java.net.HttpURLConnection;
import java.io.InputStream;

public class Tester {

    public static void main(String argv[]) throws Exception{
        InputStream is = null;

        try {
            String httpUrl = "http://httpstat.us/301";
            URL resourceUrl = new URL(httpUrl);
            HttpURLConnection conn = (HttpURLConnection)resourceUrl.openConnection();
            conn.setConnectTimeout(15000);
            conn.setReadTimeout(15000);
            conn.connect();
            is = conn.getInputStream();
            System.out.println("Original URL: "+httpUrl);
            System.out.println("Connected to: "+conn.getURL());
            System.out.println("HTTP response code received: "+conn.getResponseCode());
            System.out.println("HTTP response message received: "+conn.getResponseMessage());
       } finally {
            if (is != null) is.close();
        }
    }
}

The output of this program is:

Original URL: http://httpstat.us/301
Connected to: http://httpstat.us/301
HTTP response code received: 301
HTTP response message received: Moved Permanently

A request to http://httpstat.us/301 returns the following (shortened) response (which seems absolutely right!):

HTTP/1.1 301 Moved Permanently
Cache-Control: private
Content-Length: 21
Content-Type: text/plain; charset=utf-8
Location: https://httpstat.us

Unfortunately, Java's HttpURLConnection does not follow the redirect!

Note that if you change the original URL to HTTPS (https://httpstat.us/301), Java will follow the redirect as expected!?

Solution 1:

Redirects are followed only if they use the same protocol. (See the followRedirect() method in the source.) There is no way to disable this check.

Even though we know it mirrors HTTP, from the HTTP protocol point of view, HTTPS is just some other, completely different, unknown protocol. It would be unsafe to follow the redirect without user approval.

For example, suppose the application is set up to perform client authentication automatically. The user expects to be surfing anonymously because he's using HTTP. But if his client follows HTTPS without asking, his identity is revealed to the server.

Solution 2:

HttpURLConnection by design won't automatically redirect from HTTP to HTTPS (or vice versa). Following the redirect may have serious security consequences. SSL (hence HTTPS) creates a session that is unique to the user. This session can be reused for multiple requests. Thus, the server can track all of the requests made from a single person. This is a weak form of identity and is exploitable. Also, the SSL handshake can ask for the client's certificate. If sent to the server, then the client's identity is given to the server.

As erickson points out, suppose the application is set up to perform client authentication automatically. The user expects to be surfing anonymously because he's using HTTP. But if his client follows HTTPS without asking, his identity is revealed to the server.

The programmer has to take extra steps to ensure that credentials, client certificates or SSL session id will not be sent before redirecting from HTTP to HTTPS. The default is to send these. If the redirection hurts the user, do not follow the redirection. This is why automatic redirect is not supported.

With that understood, here's the code which will follow the redirects.

  URL resourceUrl, base, next;
  Map<String, Integer> visited;
  HttpURLConnection conn;
  String location;
  int times;

  ...
  visited = new HashMap<>();

  while (true)
  {
     times = visited.compute(url, (key, count) -> count == null ? 1 : count + 1);

     if (times > 3)
        throw new IOException("Stuck in redirect loop");

     resourceUrl = new URL(url);
     conn        = (HttpURLConnection) resourceUrl.openConnection();

     conn.setConnectTimeout(15000);
     conn.setReadTimeout(15000);
     conn.setInstanceFollowRedirects(false);   // Make the logic below easier to detect redirections
     conn.setRequestProperty("User-Agent", "Mozilla/5.0...");

     switch (conn.getResponseCode())
     {
        case HttpURLConnection.HTTP_MOVED_PERM:
        case HttpURLConnection.HTTP_MOVED_TEMP:
           location = conn.getHeaderField("Location");
           location = URLDecoder.decode(location, "UTF-8");
           base     = new URL(url);               
           next     = new URL(base, location);  // Deal with relative URLs
           url      = next.toExternalForm();
           continue;
     }

     break;
  }

  is = conn.openStream();
  ...

Solution 3:

Has something called HttpURLConnection.setFollowRedirects(false) by any chance?

You could always call

conn.setInstanceFollowRedirects(true);

if you want to make sure you don't affect the rest of the behaviour of the app.

Solution 4:

As mentioned by some of you above, the setFollowRedirect and setInstanceFollowRedirects only work automatically when the redirected protocol is same . ie from http to http and https to https.

setFolloRedirect is at class level and sets this for all instances of the url connection, whereas setInstanceFollowRedirects is only for a given instance. This way we can have different behavior for different instances.

I found a very good example here http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/

Solution 5:

Another option can be to use Apache HttpComponents Client:

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
</dependency>

Sample code:

CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("https://media-hearth.cursecdn.com/avatars/330/498/212.png");
CloseableHttpResponse response = httpclient.execute(httpget);
final HttpEntity entity = response.getEntity();
final InputStream is = entity.getContent();