Nginx uWSGI responses truncated
After double-checking that the uwsgi
command I was testing against matched all the options I'd provided in my .ini
file, I realized that my .ini
file contained processes = 5
while the uwsgi
command I was testing with did not. If I add --processes=5
to my uwsgi
command, I can reproduce the truncation issue on every hit, not just on every second request. Each time I start the uwsgi
server with --processes=5
, the first request succeeds, the second yields a 500, and all subsequent requests are truncated to 9.99MB with the following error in the console:
[2017-01-08 04:16:08,959] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "./flaskapp.py", line 13, in main
flag = dont_tread_on_memes.dont_me(caption)
File "./dont_tread_on_memes/__init__.py", line 30, in dont_me
return tread_on("don't {} me".format(phrase))
File "./dont_tread_on_memes/__init__.py", line 16, in tread_on
flag = BLANK_FLAG.copy()
File "/usr/local/lib/python3.4/dist-packages/PIL/Image.py", line 1010, in copy
self.load()
File "/usr/local/lib/python3.4/dist-packages/PIL/ImageFile.py", line 226, in load
"(%d bytes not processed)" % len(b))
OSError: image file is truncated (0 bytes not processed)
My suspicion is that this is an issue with Pillow and the way uwsgi
handles threading. Perhaps the every-other-request behavior was due to the way uwsgi
decides to spawn new processes and kill old processes, or due to caching by NGINX. Either way, I've fixed the truncation issue.
I also found this on StackOverflow from someone with my same issue.
If anyone else can provide an answer for why this occurred, or an answer as to how I can both keep this resolved and can let uwsgi
spawn multiple processes, I'd certainly consider that a more complete answer and accept it.
This almost always means that there is a problem with your images (i.e., there is missing data at the bottom). I have processed images > 25mb using PIL (provided you have enough RAM) and it works fine. The workaround here might work for you. Copy & pasted for ease of reading:
if img and img.meta_type == 'Image':
pilImg = PIL.Image.open( StringIO(str(img.data)) )
elif imgData:
pilImg = PIL.Image.open( StringIO(imgData) )
try:
pilImg.load()
except IOError:
pass # You can always log it to logger