Why does Unicorn need to be deployed together with Nginx?

Nginx is a pure web server that's intended for serving up static content and/or redirecting the request to another socket to handle the request.

Unicorn is a Rack web server and only intended to host a 'Rack App' which is usually generating dynamic content. Rack apps can also serve up static content but it's less efficient than most other traditional web servers.

Most RoR setups use a combination of both traditional web servers and Rack servers to apply the best of both of their capabilities. Nginx is incredibly fast at request redirection through proxy balancing and serving up static content. Unicorn is quite capable of processing HTTP headers and balancing inbound requests to Ruby for processing.

This answer is complementary to the other ones and explains why Unicorn needs nginx in front of it.

TL;DR The reason that Unicorn is usually deployed together with a reverse proxy like nginx is because its creators deliberately designed it so, making a tradeoff for simplicity.

First of all, there's nothing keeping you from deploying Unicorn without a reverse proxy. However, that wouldn't be a very good idea; let's see why.

Unicorn follows the Unix philosophy which is to do one thing and do it well, and that is to serve fast, low-latency clients (we'll see what this means later on). The fact that Unicorn is designed for fast, low-latency clients also implies that it's not very good with slow, high-latency clients, which is indeed true. This is one of Unicorn's weak points and it's where a reverse proxy comes into play: it sits in front of Unicorn and takes care of those slow clients (we'll see how later on).

Fortunately, such a reverse proxy already exists and is called nginx.

The decision to handle only fast clients, greatly simplifies the design of Unicorn and allows a much simpler and smaller codebase, at the cost of some added complexity on the deployment department (ie. you have to deploy nginx too in addition to Unicorn).

An alternative decision could be designing Unicorn in such a way that it wouldn't need a reverse proxy. However, this means that it would have to implement extra functionality to do all the things that now nginx does, resulting in a more complex codebase and more engineering efforts.

Instead its creators made the decision to leverage existing software that is battle-tested and very well designed and to avoid wasting time and energy on problems already solved by other software.

But let's get technical and answer your question:

Why does Unicorn needs to be deployed together with nginx?

Here are some of the key reasons:

Unicorn uses blocking I/O for clients

Relying on a reverse proxy means that Unicorn doesn't need to use non-blocking I/O. Instead it can use blocking I/O which is inherently simpler and easier for the programmer to follow.

Also as the DESIGN document states:

[Using blocking I/O] allows a simpler code path to be followed within the Ruby interpreter and fewer syscalls.

However, this also has some consequences:

Key point #1: Unicorn is not efficient with slow clients

(For simplicity's sake, we assume a setup with 1 Unicorn worker)

Since blocking I/O is used, a Unicorn worker can only serve one client at a time, so a slow client (ie. one with a slow connection) would effectively keep the worker busy for a longer time (than a fast client would do). In the meantime, the other clients would just wait until the worker is free again (ie. requests would pile up in the queue).

To get around this issue, a reverse proxy is deployed in front of Unicorn, that fully buffers incoming requests and the application responses, and then sends each of them at once (aka spoon-feeds them) to Unicorn and the clients, respectively. In that regard, you could say that the reverse proxy "shields" Unicorn from slow network clients.

Fortunately Nginx is a great candidate for this role, since it is designed to handle thousands of hundreds concurrent clients efficiently.

It's of crucial importance that the reverse proxy should be within the same local network as Unicorn (typically in the same physical machine communicating w/ Unicorn via a Unix domain socket), so that the network latency is kept to a minimum.

So such a proxy effectively plays the role of a fast client that Unicorn is designed to serve in the first place, since it proxies requests to Unicorn fast and keeps the workers busy for the shortest possible amount of time (compared to how much time a client with a slow connection would do).

Key point #2: Unicorn does not support HTTP/1.1 keep-alive

Since Unicorn uses blocking I/O, it also means that it can't support the HTTP/1.1 keep-alive feature, since the persistent connections of slow clients would quickly occupy all the available Unicorn workers.

Therefore to leverage HTTP keep-alive, guess what: a reverse proxy is used.

nginx on the other hand, can handle thousands of concurrent connections using just a few threads. Therefore, it doesn't have the concurrency limits a server like Unicorn has (which essentially limited to the amount of worker processes), which means it can handle persistent connections just fine. More of how this actually works can be found here.

That's why nginx accepts keep-alive connections from the clients and proxies them to Unicorn over plain connections via typically a Unix socket.

Point #3: Unicorn is not very good at serving static files

Again, serving static files is a thing that Unicorn can do but is not designed to do efficiently.

On the otherhand, reverse proxies like nginx though are much more better at it (ie. sendfile(2) & caching).

More

There are other points which are outlined in the PHILOSOPHY document (see "Improved Performance Through Reverse Proxying").

See also some of nginx's basic features.

We see that by leveraging existing software (ie. nginx) and following the Unix philosophy of "doing one thing and do it well", Unicorn is able to follow a simpler design and implementation while maintaining to be efficient at serving Rack apps (eg. your Rails app).

For more information refer to Unicorn's philosophy and design documents which explain in more detail the choices behind Unicorn's design and why nginx is considered a good reverse-proxy for Unicorn.

Nginx

Unicorn

Refer to unicorn on github for more information.

Nginx can be used to serve slow clients on a unicorn server as slow clients would choke the unicorn server. Nginx is used as some sort of proxy buffering all requests and responses to slow clients.

See http://unicorn.bogomips.org/