uWSGI won't start with systemd under Ubuntu 16.04

I'm in the process of doing a somewhat painful migration of our staging and production servers from Ubuntu 12.04 to 16.04. I'm testing the migration on staging and it is mostly working, except for getting uWSGI to start under systemd (it previously worked fine under Upstart). This works without a problem:

uwsgi --ini /etc/uwsgi/my_wsgi.ini

But running the following doesn't work (uWSGI doesn't start, but doesn't produce an error):

sudo systemctl start uwsgi

I created the following service in /etc/systemd/system/uwsgi.service:

[Unit]
Description=uWSGI Service

[Service]
ExecStart=/usr/local/bin/uwsgi --ini /etc/uwsgi/my_wsgi.ini
Restart=always
RestartSec=5
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all

[Install]
WantedBy=multi-user.target

and my_wsgi.ini has the following:

[uwsgi]

# Django-related settings
# the base directory (full path)
chdir           = /path/to/project/hidden
# Django's wsgi file
module          = wsgi
# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 8
# the socket (use the full path to be safe)
socket          = /path/to/socket/hidden
# ... with appropriate permissions - may be needed
chmod-socket    = 666
# clear environment on exit
vacuum          = true
# Mercilessly kill this worker if it takes longer than this time to reload
reload-mercy    = 8
# Reload workers after the specified amount of managed requests (avoid memory leaks)
max-requests    = 5000
# Every request that will take longer than the seconds specified in the harakiri timeout will be dropped and the corresponding worker is thereafter recycled.
harakiri        = 90
# Log everything and make daemon
daemonize       = /var/log/uwsgi/my.log
# use custom settings file
env             = DJANGO_SETTINGS_MODULE=settings.staging
# set pid file for starting/stopping server
pidfile         = /path/to/pid/hidden.pid
# See https://docs.newrelic.com/docs/python/python-agent-and-uwsgi
enable-threads  = true
single-interpreter = true

Running systemd-analyze verify uwsgi.service returns nothing and sudo journalctl returns the following:

Starting uWSGI Service...
[uWSGI] getting INI configuration from /etc/uwsgi/my_wsgi.ini
Started uWSGI Service.
uwsgi.service: Service hold-off time over, scheduling restart.
Stopped uWSGI Service.
... (repeats a few times) ....

I suspect that uwsgi.service: Service hold-off time over, scheduling restart. might point to the issue, but I haven't been able to figure it out.

Solution 1:

The uwsgi documentation clearly states:

NOTE: DO NOT daemonize the Emperor (or the master) unless you know what you are doing!!!

Unfortunately, you daemonized the Emperor without knowing what you are doing. From your question:

# Log everything and make daemon
daemonize       = /var/log/uwsgi/my.log

The problem is that uwsgi is systemd-aware, and the systemd unit tells systemd that uwsgi will notify systemd when uwsgi is ready to begin processing requests. By default uwsgi starts in this mode when on a systemd system, unless you tell it to daemonize. In that case, uwsgi does not talk to systemd. This is why systemd waited for a while and finally gave up waiting for uwsgi to tell systemd that it was ready.

Rather than having uwsgi daemonize and log directly to a file, the documentation suggests that you should let it log to its default of standard error, and let systemd pick up the logs and log them. You'll want to do this if you eventually will have logging sent elsewhere, such as an ELK stack. For example:

[Service]
StandardError=syslog

Or you can log directly from uwsgi without daemonizing.

logto = /var/log/uwsgi/my.log