Tools for introspecting Varnish

Solution 1:

varnishncsa displays logs in Apache/NCSA combined log format (from man page). You can use these with any normal log analyzer to find which user agents you have, which pages they load etc.

varnishlog (/usr/bin/varnishlog) on the other hand displays detailed output for all the requests, containing information like which VCL commands are being used (lookup, hash, hit, deliver etc.) and all headers in the HTTP transaction (varnishlog -o will group them by request). If you have a busy site this will scroll through very quickly, but you can write them to disk using -w, or use the logger facility (/etc/init.d/varnishlog). Be aware that this quickly fills up the disk, because it is very detailed (10-30 lines per request depending on where the requests get sent, and of course all your HTTP headers).

varnishreplay can replay the HTTP traffic from such a log file.

Most of the tools can be used with regex searching.

Redpill Linpro, the company making Varnish, are currently making a web gui. I'm not sure which features it will have (I don't think they quite know either), but I am assured that it will have quite a few monitoring graphs.

Solution 2:

• try varnishncsa to get per request logging
• varnish top has avg service and hit times
• IMHO, if you suspect problems with your backends you should be monitoring the directly, without going through varnish.

Solution 3:

varnishncsa:

One trick is to use a custom log format with varnishncsa to include a few more pieces of useful information such as:

• Response time (to first byte)
• Hit or miss status for the request
• The original client IP (if Varnish is behind a load-balancer)

I use the following in my /etc/varnish/varnishncsa to add this information (varnishncsa documentation):

#! /bin/sh
DAEMON_OPTS="$DAEMON_OPTS -F \"%h \\\"%{X-Forwarded-For}i\\\" %t \\\"%r\\\" %s %b \\\"%{Referer}i\\\" \\\"%{User-agent}i\\\" %{Varnish:time_firstbyte}x %{Varnish:hitmiss}x\""

This configuration allows me to grep through the access logs and add up historic hit-rates as well as find pages that respond slowly.

varnishlog:

As discussed in this question/answer, if you are having varnish perform health-checks on your back-ends you can use varnishlog to record changes in back-end health. Running varnishlog -O -I "healthy|sick" will give you a running update of back-end health based on health-checks.