IOError: request data read error

Solution 1:

I get this exception, too. In the Apache error logfile I see this:

[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] (70014)End of file found: mod_wsgi (pid=9722): Unable to get bucket brigade for request., referer: https://egs-work/modwork/beleg/188074/edit/
[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] mod_wsgi (pid=3572): Exception occurred processing WSGI script '/home/modwork_egs_p/modwork_egs/apache/django_wsgi.py'.
[Wed Aug 17 08:30:45 2011] [error] [client 10.114.48.206] IOError: failed to write data

Versions:

apache2-prefork-2.2.15-3.7.x86_64
apache2-mod_wsgi-3.3-1.8.x86_64 WSGIDaemonProcess with threads=1
mod_ssl/2.2.15
Linux egs-work 2.6.34.8-0.2-default #1 SMP 2011-04-06 18:11:26 +0200 x86_64 x86_64 x86_64 GNU/Linux
openSUSE 11.3 (x86_64)

First I was confused, because the last line "failed to write data" does not fit to the django code "load post data". But I guess that django wants to write an error page to the client. But the client has canceled the tcp connection. And now http 500 page can't be written to the client.

The client disconnected after sending the request, and before getting the response:

• The user closed the browser or navigated to an other page.
• The user pressed the reload button.

I have seen this only with POST-Requests (not GET). If POST is used, the webserver does read at least twice: First to get the headers, the second to get the data. The second read fails.

It is easy to reproduce:

Insert some code which waits before the first access to request.POST happens (be sure, that no middleware accesses request.POST before time.sleep()):

def edit(request):
    import time
    time.sleep(3)
    #.....

Now do a big POST (e.g. file upload). I don't know the apache buffer size. But 5 MB should be enough. When the browser shows the hourglass, browse to an other page. The browser will cancel the request and the exception should be in the logfile.

This is my Middleware, since I don't want to get the above traceback in our logfiles:

class HandleExceptionMiddleware:

    def process_exception(self, request, exception):
        if isinstance(exception, IOError) and 'request data read error' in unicode(exception):
            logging.info('%s %s: %s: Request was canceled by the client.' % (
                    request.build_absolute_uri(), request.user, exception))
            return HttpResponseServerError()

Solution 2:

as you might think, this is no django error.

see https://groups.google.com/group/django-users/browse_thread/thread/946936f69c012d96

have the error myself (but IE ajax requests only, no file upload, just post data).

will add an complete answer if i ever find out how to fix this.

Solution 3:

We were seeing this error on uploads to Django Rest Framework when the content-type header was incorrectly set to application/json. The post was actually multipart form data. The errors stopped when we removed the incorrect content-type header.

Solution 4:

This happened to me recently. I was using django-ajax-uploader and small files were uploading successfully but large files e.g. 100MB were breaking in between with IOError: request data read error.

I checked my Apache configuration and found these setting RequestReadTimeout header=90 body=90 which means Allow 90 seconds to receive the request including the headers and 90 seconds for receiving the request body.

File is received in chunks at backend, which means if file size is big 90 seconds are not enough for some uploads. So how to determine the best value (seconds) for the requests?

So I have used this setting:

RequestReadTimeout header=90,MinRate=500 body=90,MinRate=500

Defining the MinRate solves the issue for me. The above setting states that:

Allow at least 90 seconds to receive the request body. If the client sends data, increase the timeout by 1 second for every 500 bytes received

As the client is sending data continuously (ajax upload) it makes sense to automatically increase the timeout if data is received. More information/variations about the RequestReadTimeout can be found here.

Solution 5:

Taking this from the thread: Getting rid of Django IOErrors

Extending the possible solution by @dlowe for Django 1.3, to suppress the IOError in concern, we can write the full working example as:

settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'supress_unreadable_post': {
            '()': 'common.logging.SuppressUnreadablePost',
        }
    },
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
            'filters': ['supress_unreadable_post'],
        }
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
    }
}

common/logging.py

import sys, traceback

class SuppressUnreadablePost(object):
    def filter(self, record):
        _, exception, tb = sys.exc_info()
        if isinstance(exception, IOError):
            for _, _, function, _ in traceback.extract_tb(tb):
                if function == '_get_raw_post_data':
                    return False
        return True