What is the use for IHttpHandler.IsReusable?

This property indicates if multiple requests can be processed with the same IHttpHandler instance. By default at the end of a request pipeline all http handlers that are placed in the handlerRecycleList of the HttpApplication are set to null. If a handler is reusable it will not be set to null and the instance will be reused in the next request.

The main gain is performance because there will be less objects to garbage-collect.
The most important pain-point for reusable handler is that it must be thread-safe. This is not trivial and requires some effort.

I personally suggest that you leave the default value (not reusable) if you use only managed resources because the Garbage Collector should easily handle them. The performance gain from reusable handlers is usually negligible compared to the risk of introducing hard to find threading bugs.

If you decide to reuse the handler you should avoid maintaining state in class variables because if the handler instance is accessed concurrently multiple requests will write/read the values.

Apparently, this keeps the handler in memory and able to handle multiple requests. When set to false, it has to create a new instance of the handler for each incoming request.

Here's a question that shows what happens when it's not used properly:

Streaming Databased Images Using HttpHandler

It's cheaper to recycle the handler than to new one up every time a request comes in and the server will chum less memory, easing the work GC has to perform. If the handler is in a state where dealing with a new request would not be problematic (i.e. any state in the handler instance has been reset), then it should qualify as being reusable.

EDIT

I'm not sure if my answer correctly defines what reuse is. It actually allows for concurrent reuse, so effectively state would be best avoided or carefully managed in a thread-safe manner.