Nginx and fcgiwrap, incremental output problems

Solution 1:

Ok, figured it out. It seems both nginx and fcgiwrap have undesirable buffering in place. For nginx it seems it can be disabled through configuration while fcgiwrap needs a patch.

I have posted a debdiff for fcgiwrap at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863478 this adds a new option NO_BUFFERING

(update: Debian has now applied said patch, so if you are running Debian Buster or later you don't need to apply it yourself).

Then the nginx configuration needs changing to pass the NO_BUFFERING option to fcgiwrap and to disable buffering inside nginx.

To do this I added some settings immediately before and after the "include fastcgi_params;" line.

    #note: NO_BUFFERING relies on a patched fcgiwrap.
    fastcgi_param NO_BUFFERING 1;
    include fastcgi_params;
    gzip off;
    fastcgi_buffering off;