Nginx Varnish Nginx Django?

Solution 1:

We are talking 1 - 3 frontend servers in total, not a large server farm with load balancing between the tiers?

Putting nginx in front of Vanish enables you to do HTTP compression on the fly. That is a performance best practice, but it could be dispensed with. (Content in Varnish is often kept uncompressed, so that ESI Includes work, and so you don't have to deal with multiple cached versions of the same object depending on Vary header / browser matching.)

Regarding nginx on the app server -- is Apache with mod_wsgi not the recommended and most common way to deploy new Django installations nowadays? I'm not aware of a compelling reason for using nginx/fastcgi over Apache/mod_wsgi for Django; but you should get advice from a Django expert.

Regarding Varnish having attractive load balancing features that nginx doesn't have, I don't see what they are? Varnish has random and round-robin balancing. nginx has round-robin, client IP and consistent hashing -- I don't see a significant benefit for Varnish? Is it VCL or Varnish' graceful config reload or something else?

For a small 1-3 server setup I guess I would just do

Varnish --> Apache / mod_wsgi / Django

or maybe

Squid --> Apache / mod_wsgi / Django

and ignore HTTP compression for simplicity, unless bandwith is expensive.

Update:

Graham Dumpleton has written a valuable comment below. He mentions a very common setup for for example a blog on a VPS, or a small web farm without caching:

nginx --> Apache / mod_wsgi / Django

This is a very good solution, for a couple of reasons:

1. Simple setup
2. nginx, which has high speed and minimal overhead, handles static file serving and browser connection keepalive.
3. Django runs in Graham Dumpleton's excellent mod_wsgi, the recommended platform for Django.

The reason I didn't mention this initially is that OP seemed to require Varnish, a very high performance caching solution. The nginx / Apache / mod_wsgi combo can not do caching with a level of performance and flexibility that matches Varnish.

Solution 2:

You can use nginx without varnish to proxy and cache the content.

Solution 3:

I've being using Nginx, Varnish, and Apache/mod_wsgi/Django successfully. I started with the following config:

Nginx -> Apache/mod_wsgi/Django

Once I started seeing significant load on Apache, I added Varnish:

Nginx -> Varnish -> Apache/mod_wsgi/Django

I use Nginx as a kind of "URL router". Django admin requests are sent directly from Nginx to Apache. Client requests are sent from Nginx to Varnish which caches the requests from Apache and also serves "graced" items from the cache if the app servers are unavailable.

My Nginx server also serves certain static content directly (e.g. images, CSS, and javascript files).

In general, performance has been excellent. I have noticed a couple of caveats that I should mention:

1. On a busy site, restarting Varnish can cause the load to spike on the app servers, so it's best to keep Varnish restarts to a minimum. (Varnish doesn't seem to have a "reload" like Nginx/Apache where it just rereads its VCL files). Conversely, reloading an Nginx configuration has minimal impact. For this reason, I do most of the URL rewriting and "routing" in Nginx.
2. Varnish is easy to drop in between Nginx and Apache. If you start noticing high load on your app servers, adding varnish with even the default config can really make a difference.
3. If you do use Varnish, you definitely need to think about how you're going to handle cache invalidation.
4. My experience has been that Varnish handles failed backends a bit more gracefully than Nginx (as you pointed out earlier).