Force web address to go through HTTPS

I wonder if all third-party applications will actually take proxy settings into account. Or if changing system-wide proxy settings might interfere with other applications. (Like when the company's proxy is required for internet access, so cannot be changed just to redirect traffic for twitter.com.) Luckily, when the application does not use HTTPS, then it cannot find a man-in-the-middle. So: set up a man-in-the-middle for twitter.com on port 80, using DeleGate.

The following steps have been tested on Mac OS X 10.6 and Windows XP, using accounts with full administrative rights.

1. Download DeleGate. Don't let the 1990's homepage fool you: the program is still maintained.

2. Tell DeleGate to forward all local requests on port 80 (and 443) to the HTTPS server, based on the value of Host header in the HTTP request. Like for a Mac on Intel (where sudo is required to use privileged ports below 1024):

   sudo ./macosxi-dg -v -P80,443 \
   SERVER=https \
   RELAY=vhost \
   RESOLV=cache,dns \
   STLS=-fcl,fsv \
   [email protected]
   

   For Windows, if unzipped to c:\:

   cd c:\dg9_9_4\bin
   dg9_9_4.exe -v -P80,443 SERVER=https RELAY=vhost RESOLV=cache,dns STLS=-fcl,fsv [email protected]
   

   If you're required to use your company's proxy for internet access, then DeleGate will happily use that if you add something like PROXY=proxy.example.com:8080 to the command line.

3. In your /etc/hosts file (c:\windows\system32\drivers\etc\hosts on Windows XP; see Wikipedia for locations on other OS's), add the following line to direct all requests for twitter.com to your own computer. Note that the mapping from domain name to IP address does not take the protocol into account. So: this will not only be used for HTTP, but also for HTTPS (and everything else, such as commands like ping).

   127.0.0.1 twitter.com
   

4. Ensure your browser is not set to use a proxy server, or add twitter.com as an exception. Just in case your browser has cached Twitter's IP address, you might want to restart it.

5. Now, http://twitter.com actually gets you (and all your applications) https://twitter.com.

The output shows that indeed the HTTPS site is requested from https://twitter.com:

REQUEST - GET / HTTP/1.1
REQUEST = https://twitter.com:443/ GET / HTTP/1.1
[..]
ConnectToServer connected [16] {168.143.161.20:443 <- 192.168.1.68:57067}
## SSLway -- TLSxSNI: sent ru=0 ty=0 nm=localhost
## SSLway ## 0.459622 connected/accepted
## SSLway server's cert. = 
  **subject /
  C=US/
  O=twitter.com/
  OU=GT09721236/
  OU=See www.rapidssl.com/resources/cps (c)09/
  OU=Domain Control Validated - RapidSSL(R)/
  CN=twitter.com
  **issuer /
  C=US/
  O=Equifax Secure Inc./
  CN=Equifax Secure Global eBusiness CA-1

When using MOUNT="/* https://twitter.com/*" instead of RELAY=vhost then even http://localhost would give one https://twitter.com:

When explicitly requesting HTTPS using https://twitter.com, then the trusted certificate chain is broken: a HTTPS-aware applicate will discover the man-in-the-middle attack, and will fail if it cannot ask you for your permission to continue:

After testing, to run as a service on Windows, simply remove the -v parameter. This will install the program as a service. It will then run in the background, and ask you if you want to run it on startup:

Trying to start as a service [DeleGate Server -P80,443] ...
Set Automatic Start on System Startup ? [y] / n :

After running the above command without the -v parameter: see Control Panel » Administrative Tools » Services to manually start or stop DeleGate. Note that this service will refer to the location from which you initially started the dg9_9_4.exe program. So, you should not delete or move that program; be sure to unzip the download to, for example, c:\dg9_9_4 to avoid a reference to some Downloads directory that you might delete in the future.

To remove the service, just ensure to specify the same value for the -P parameter:

dg9_9_4.exe -P80,443 [email protected]
[..]
The service `DeleGate Server -P80,443' exists.  Delete it ? [y] / n : y
OK. DELETEd the previous service.
Create a new service ? [y] / n : n

Finally, one may wonder how DeleGate knows the IP address of twitter.com (as we've mapped that to 127.0.0.1 in the hosts file). DeleGate actually retrieves that itself, because of RESOLV=cache,dns:

MOUNT[5]X[2] /* https://twitter.com/*  
{R} SOA got [162.143.168.in-addr.arpa][ns1.dn.net]
  [dnsadmin.enterprise.verio.net] 2008121001 10800 3600 604800 86400

You need some kind of proxy server with a rewrite rule capability. I suspect that, in the absence of a better answer being offered, you could write a modifier rule for Fiddler to do this. Fiddler rules are written in JavaScript and there are several examples online.

Edit: Please see, and vote up, Eric Law's answer on this page.

The Fiddler Rule is pretty simple. Inside OnBeforeRequest, add:

if (oSession.fullUrl.StartsWith("http://twitter.com/"))
{
    oSession.oRequest.headers.UriScheme = "https";
}

You could escape all censure by using a VPN and tunneling out of your work network.

VPN tunneling involves establishing and maintaining a logical network connection (that may contain intermediate hops). On this connection, packets constructed in a specific VPN protocol format are encapsulated within some other base or carrier protocol, then transmitted between VPN client and server, and finally de-encapsulated on the receiving side.

What it means is that all your packets are sent via an intermediary on the internet. This intermediary will normally not be blocked by your company's firewall, and it forwards all your packets to their destination, and then returns the received answers to your computer. In addition, the VPN connection ensures that all your packets both ways are encoded and secure from prying eyes.

You will get the best service from commercial products, but some are free and not too bad. For example, HTTP-Tunnel Client is a very good product.

It's not free, but the Charles (a web debugging proxy) has a Map Remote Settings function which could be told to map http://twitter.com to https://twitter.com easily using a GUI. You can download a trial version from the site to see if it meets your needs.