HAProxy health checks: using httpchk and observe?

The distinction I understand now is that /check.txt does actually return a 200 response but all requests to the application return a 500. HAProxy sees the 500s coming back from the proxied requests and takes the server out of the pool but then initiates its own check, receives a 200 and puts the server back in the pool.

The solution would be to do one of:

1. Configure Apache, rather than the application, so that every request returns a 500 response, even the static file /check.txt.
2. Change /check.txt into a Ruby app that contains just enough logic to choose between a 200 and a 500 response when appropriate.
3. Set the inter value to something ridiculous like 3600. This should give you an hour to do your testing or (if the server went down on its own) figure out the problem and bring it back up.
4. Set the inter value to something smaller like 60 but set the rise value to something higher like 60. This would also give you an hour before the server was added back to the pool. (Note, these two are listed last because they're probably very bad ideas.)