How to reproduce http traffic of a real site in another environment (e.g. VM)
There is a bug in my web application which I cannot reproduce. All the logs look ok or at least I cannot see anything unusual. But it happens. So I thought I could record all the traffic to/from my web site waiting for the bug to happen and having that, replay it somehow in my testing environment. And it did happen! So I have the data captured by this:
tcpdump -s 1514 -X tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0) -Z root -C 20 -W 1000 -w capture.cap
and I don't know what can I do with it. I have a virtual machine running the same version of the web app and I'd like to replay the recorded data in it. How should I approach this?
[EDIT]
I tried doing this with tcpreplay following this example but my web server has not logged any traffic.
My testing machine (Fedora 8) runs in VMWare Player. Assuming that the IP of the machine is 192.168.41.128 and it has one network interface: eth0 (except loopback) - how the steps #2 and #3 from the example should look like in my case? Should I run it on the same machine or from another one?
Solution 1:
You have a few options, all of them cool and none of them drop-in products. Some assembly required.
- There's the PERL application Sprocket to be able to replay web traffic. Here's the raw PERL.
- There's TCPReplay.
- If you use a Varnish server, you can use varnishreplay.
- There's also a tool called "PReplay" that is for Windows machines.
- JWireReplay will replay tcpdump captures.
What you will likely want to do is forget the logs you've already inspected, or at least abandon any desire to replay those files. Set up one of these systems and use it in a controlled manner to test out your traffic.
Solution 2:
You can use tcptrace to convert the pcap file into separate requests which can then be replayed using netcat. Here is an example:
tcpdump -s 0 port http -i en0 -w dump.pcap
tcptrace -e dump.pcap
cat *.dat | nc -v <IP_OF_YOUR_TEST_HOST>
The second step might create too many files for * to work correctly. In this case, use a for loop for the third command:
for file in $(ls | grep .dat); do cat $file | nc -v <IP_OF_YOUR_TEST_HOST>; done
Note that this might be slower than your original traffic.
Solution 3:
GoReplay is designed for capturing production traffic and replaying that in test environments. See https://goreplay.org/ or https://github.com/buger/goreplay/wiki for more information.