How to enable wire logging for a java HttpURLConnection traffic?
According to Sun's HttpURLConnection source there is some logging support via JUL.
Setup (adjust path as required):
-Djava.util.logging.config.file=/full/path/to/logging.properties
logging.properties:
handlers= java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level = FINEST
sun.net.www.protocol.http.HttpURLConnection.level=ALL
This will log to the console, adjust as required to e.g. log to a file.
Example output:
2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection writeRequests
FIN: sun.net.www.MessageHeader@16caf435 pairs: {GET /howto.html HTTP/1.1: null}{User-Agent: Java/1.6.0_20}{Host: www.rgagnon.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection getInputStream
FIN: sun.net.www.MessageHeader@5ac0728 pairs: {null: HTTP/1.1 200 OK}{Date: Sat, 07 Aug 2010 04:00:33 GMT}{Server: Apache}{Accept-Ranges: bytes}{Content-Length: 17912}{Keep-Alive: timeout=5, max=64}{Connection: Keep-Alive}{Content-Type: text/html}
Note that this prints only headers without body.
See http://www.rgagnon.com/javadetails/java-debug-HttpURLConnection-problem.html for details.
There is also system property -Djavax.net.debug=all. But it's mainly useful for SSL debugging.
I've been able to log all SSL traffic implementing my own SSLSocketFactory on top of the default one.
This worked for me because all of our connections are using HTTPS and we can set the socket factory with the method HttpsURLConnection.setSSLSocketFactory.
A more complete solution that enables monitoring on all sockets can be found at http://www.javaspecialists.eu/archive/Issue169.html Thanks to Lawrence Dol for pointing in the right direction of using Socket.setSocketImplFactory
Here is my not ready for production code:
public class WireLogSSLSocketFactory extends SSLSocketFactory {
private SSLSocketFactory delegate;
public WireLogSSLSocketFactory(SSLSocketFactory sf0) {
this.delegate = sf0;
}
public Socket createSocket(Socket s, String host, int port,
boolean autoClose) throws IOException {
return new WireLogSocket((SSLSocket) delegate.createSocket(s, host, port, autoClose));
}
/*
...
*/
private static class WireLogSocket extends SSLSocket {
private SSLSocket delegate;
public WireLogSocket(SSLSocket s) {
this.delegate = s;
}
public OutputStream getOutputStream() throws IOException {
return new LoggingOutputStream(delegate.getOutputStream());
}
/*
...
*/
private static class LoggingOutputStream extends FilterOutputStream {
private static final Logger logger = Logger.getLogger(WireLogSocket.LoggingOutputStream.class);
//I'm using a fixed charset because my app always uses the same.
private static final String CHARSET = "ISO-8859-1";
private StringBuffer sb = new StringBuffer();
public LoggingOutputStream(OutputStream out) {
super(out);
}
public void write(byte[] b, int off, int len)
throws IOException {
sb.append(new String(b, off, len, CHARSET));
logger.info("\n" + sb.toString());
out.write(b, off, len);
}
public void write(int b) throws IOException {
sb.append(b);
logger.info("\n" + sb.toString());
out.write(b);
}
public void close() throws IOException {
logger.info("\n" + sb.toString());
super.close();
}
}
}
}
Solution#1: Use Decorator Pattern
You will have to use Decorator pattern on HttpURLConnection class to extend it's functionality. Then override all HttpURLConnection method(s) and delegate operation to Component pointer as well as capture required information and log it.
Also make sure you override parent class URLConnection.getOutputStream() : OutputStream and URLConnection.html#getInputStream() : InputStream methods to return decorated OutputStream and InputStream objects as well.
.
Solution#2: Use custom, in-memory http proxy
Write a simple http proxy server and have it start in it's separate thread during application startup and initialization. See Example simple proxy server.
Have your application configured to use above HTTP proxy for all your requests. See configuring Java to use Proxies.
Now all your traffic is going through above proxy, just like how it happens in fiddler. Hence you have access to raw http stream "from client to server" as well as "back from server to client". You will have to interpret this raw information and log it as required.
Update: Use HTTP Proxy as Adapter to SSL based Web Server.
== Client System ===============================
| |
| ------------------------------ |
| | | |
| | Java process | |
| | ---- | |
| | ---------- | | | |
| | | | -O | | |
| | | Logging | | | |
| | | Proxy <---HTTP-- | ----- |
| | | Adapter | | | | |
| | | Thread o------------------> | |
| | | o | | | | |
| | --------|- | | Log | |
| | | | ----- |
| ----------------|------------- |
| | |
=====================|==========================
|
|
HTTPS
SSL
|
== Server System ====|==========================
| | |
| ----------------|---------------- |
| | V | |
| | | |
| | Web Server | |
| | | |
| --------------------------------- |
| |
================================================
In Linux you can run the VM under strace:
strace -o strace.out -s 4096 -e trace=network -f java ...
To refresh with Java 8 environment:
Following @sleske answer
System.setProperty("javax.net.debug","all");
worked for me out of the box.
Also was @weberjn suggestion of
strace -o strace.out -s 4096 -e trace=network -f java
but not useful if handling with SSL traffic as it dumps encoded stream.
All other code tricks did not work for me, but maybe not trying enough hard.