Varnish -> Nginx -> Apache a good idea?

Yeah, it's valid. My personal approach would be to use Varnish up front and use VCL to split the traffic between static NGINX requests and your heavy lifting (whether that be Apache or Passenger or... it doesn't matter). This is especially true if it is on the same machine as you don't need the extra overhead. It doesn't necessarily buy you anything.

Varnish doesn't (yet) support gzip compression, so it might be an idea to swap it around with nginx in front to compress what varnish sends back. Since varnish and nginx don't fight for the same resources (nginx uses CPU for gzip compression, while varnish uses memory) they should run smoothly on the same machine.

Varnish now supports gzip compression, so unless you need SSL termination (as suggested in the comments), I would suggest putting varnish directly in contact with the Internet.

For http:

(internet) --> (varnish, gzip, caching, esi) --> (application)

For https:

(internet) --> (nginx, ssl)--> (varnish, gzip, caching, esi) --> (application)

If you want apache in there too (for the ubiquitous mod_foobar support), I'd put it between varnish and the application

Update: Updated to include gzip support in varnish 3.0. Added ssl / esi as suggested in comments

The amount of overhead shouldn't be significant. I'm assuming part of the reason you want to have these two tiers is for scalability; in which case you would most likely see, relative to apache, that varnish and nginx aren't working very hard.

If you all three tiers on one machine, there should be less of a performance impact before you reach the capacity of the server itself.

As an alternative, why not varnish + nginx with passenger? I've used this setup in the past and nginx using passenger is relatively light, and ran pretty well. Might be worth thinking about if you're not married to apache running your rails stack.