Python daemon and systemd service

The reason, it does not complete the startup sequence is, that for Type forking your startup process is expected to fork and exit (see $ man systemd.service - search for forking).

Simply use only the main process, do not daemonize

One option is to do less. With systemd, there is often no need to create daemons and you may directly run the code without daemonizing.

#!/usr/bin/python -u
from somewhere import Node
check = Node()
check.run()

This allows using simpler Type of service called simple, so your unit file would look like.

[Unit]
Description=Simplified simple zebra service
After=syslog.target

[Service]
Type=simple
User=node
Group=node
WorkingDirectory=/home/node/Node/
ExecStart=/home/node/Node/node.py
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target

Note, that the -u in python shebang is not necessary, but in case you print something out to the stdout or stderr, the -u makes sure, there is no output buffering in place and printed lines will be immediately caught by systemd and recorded in journal. Without it, it would appear with some delay.

For this purpose I added into unit file the lines StandardOutput=syslog and StandardError=syslog. If you do not care about printed output in your journal, do not care about these lines (they do not have to be present).

systemd makes daemonization obsolete

While the title of your question explicitly asks about daemonizing, I guess, the core of the question is "how to make my service running" and while using main process seems much simpler (you do not have to care about daemons at all), it could be considered answer to your question.

I think, that many people use daemonizing just because "everybody does it". With systemd the reasons for daemonizing are often obsolete. There might be some reasons to use daemonization, but it will be rare case now.

EDIT: fixed python -p to proper python -u. thanks kmftzg

It is possible to daemonize like Schnouki and Amit describe. But with systemd this is not necessary. There are two nicer ways to initialize the daemon: socket-activation and explicit notification with sd_notify().

Socket activation works for daemons which want to listen on a network port or UNIX socket or similar. Systemd would open the socket, listen on it, and then spawn the daemon when a connection comes in. This is the preferred approch because it gives the most flexibility to the administrator. [1] and [2] give a nice introduction, [3] describes the C API, while [4] describes the Python API.

[1] http://0pointer.de/blog/projects/socket-activation.html
[2] http://0pointer.de/blog/projects/socket-activation2.html
[3] http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html
[4] http://www.freedesktop.org/software/systemd/python-systemd/daemon.html#systemd.daemon.listen_fds

Explicit notification means that the daemon opens the sockets itself and/or does any other initialization, and then notifies init that it is ready and can serve requests. This can be implemented with the "forking protocol", but actually it is nicer to just send a notification to systemd with sd_notify(). Python wrapper is called systemd.daemon.notify and will be one line to use [5].

[5] http://www.freedesktop.org/software/systemd/python-systemd/daemon.html#systemd.daemon.notify

In this case the unit file would have Type=notify, and call systemd.daemon.notify("READY=1") after it has established the sockets. No forking or daemonization is necessary.

You're not creating the PID file.

systemd expects your program to write its PID in /var/run/zebra.pid. As you don't do it, systemd probably thinks that your program is failing, hence deactivating it.

To add the PID file, install lockfile and change your code to this:

import daemon
import daemon.pidlockfile 

pidfile = daemon.pidlockfile.PIDLockFile("/var/run/zebra.pid")
with daemon.DaemonContext(pidfile=pidfile):
    check = Node()
    check.run()

(Quick note: some recent update of lockfile changed its API and made it incompatible with python-daemon. To fix it, edit daemon/pidlockfile.py, remove LinkFileLock from the imports, and add from lockfile.linklockfile import LinkLockFile as LinkFileLock.)

Be careful of one other thing: DaemonContext changes the working dir of your program to /, making the WorkingDirectory of your service file useless. If you want DaemonContext to chdir into another directory, use DaemonContext(pidfile=pidfile, working_directory="/path/to/dir").

Also, you most likely need to set daemon_context=True when creating the DaemonContext().

This is because, if python-daemon detects that if it is running under a init system, it doesn't detach from the parent process. systemd expects that the daemon process running with Type=forking will do so. Hence, you need that, else systemd will keep waiting, and finally kill the process.

If you are curious, in python-daemon's daemon module, you will see this code:

def is_detach_process_context_required():
    """ Determine whether detaching process context is required.

        Return ``True`` if the process environment indicates the
        process is already detached:

        * Process was started by `init`; or

        * Process was started by `inetd`.

        """
    result = True
    if is_process_started_by_init() or is_process_started_by_superserver():
        result = False

Hopefully this explains better.

I came across this question when trying to convert some python init.d services to systemd under CentOS 7. This seems to work great for me, by placing this file in /etc/systemd/system/:

[Unit]
Description=manages worker instances as a service
After=multi-user.target

[Service]
Type=idle
User=node
ExecStart=/usr/bin/python /path/to/your/module.py
Restart=always
TimeoutStartSec=10
RestartSec=10

[Install]
WantedBy=multi-user.target

I then dropped my old init.d service file from /etc/init.d and ran sudo systemctl daemon-reload to reload systemd.

I wanted my service to auto restart, hence the restart options. I also found using idle for Type made more sense than simple.

Behavior of idle is very similar to simple; however, actual execution of the service binary is delayed until all active jobs are dispatched. This may be used to avoid interleaving of output of shell services with the status output on the console.

More details on the options I used here.

I also experimented with keeping the old service and having systemd resart the service but I ran into some issues.

[Unit]
# Added this to the above
#SourcePath=/etc/init.d/old-service 

[Service]
# Replace the ExecStart from above with these
#ExecStart=/etc/init.d/old-service start
#ExecStop=/etc/init.d/old-service stop

The issues I experienced was that the init.d service script was used instead of the systemd service if both were named the same. If you killed the init.d initiated process, the systemd script would then take over. But if you ran service <service-name> stop it would refer to the old init.d service. So I found the best way was to drop the old init.d service and the service command referred to the systemd service instead.

Hope this helps!