nginx: Log complete request / response with all headers?

To get the request body sent by visitors, use client_body_in_file_only on; and log the "temporary" file it's written to in the logs by appending var $request_body_file to the log format. "Temporary" files will be located in client_temp directory by default.

You can log request headers $http_<header> too and sent headers with $sent_http_<header>.

If you have request body and headers you should be able to replay it and get the response your visitor had.

Also something like gor should highly be considered so you could replay the traffic on an other environment where you could let nginx write these temporary files without causing IO issues in production (nginx won't purge them with on value that's why It's not that "temporary" in this case).


mitmproxy seems to be the right tool to do what you are asking.

mitmproxy is an interactive, SSL-capable man-in-the-middle proxy for HTTP with a console interface.

mitmdump is the command-line version of mitmproxy. Think tcpdump for HTTP.

Features

  • Intercept HTTP requests and responses and modify them on the fly.
  • Save complete HTTP conversations for later replay and analysis.
  • Replay the client-side of an HTTP conversations. Replay HTTP responses of a previously recorded server.
  • Reverse proxy mode to forward traffic to a specified server.
  • Transparent proxy mode on OSX and Linux.
  • Make scripted changes to HTTP traffic using Python.
  • SSL certificates for interception are generated on the fly.

The reverse proxy mode would let you capture the request and response just like Fiddler does.