Is /var/run/user/$UID the new /var/run for PID files?

I have an application (running as a service as root) that creates a PID file under /var/run. But I'm wondering whether this is now no longer best practice.

In Linux - alternative places where to store pid file instead of /var/run, asked in 2012, the questioner asks whether PID files should be placed in /var/run at all. That was, though, largely prior to the advent of systemd, and the move on systemd operating systems from /var/run to just /run (as one of the systemd "API File Systems" and listed in the systemd File Hierarchy Requirements).

Things like the XDG Base Directory Specification by Lennart Poettering (and others) talk about other places for "user-specific non-essential runtime files and other file objects (such as sockets, named pipes, …)". So too does the systemd file-hierarchy manual page.

And what I've read elsewhere gives me the impression that /var/run/user/$UID is a new standard location for such things on systemd operating systems.

• The answers in https://unix.stackexchange.com/questions/162900/ talk about "files used for running processes". It suggests /var/run/user/0 would be the appropriate location if the program runs as root - but this posting also says the directory is removed/cleared when there are no more active sessions. Since this is a service, is this really the right location?
• https://github.com/systemd/systemd/issues/735#issuecomment-126309537 shows a process invoked by systemd with the command-line arguments --pid-file /run/user/1000/uzbl/event/pid
• https://lists.debian.org/debian-user/2015/10/msg01315.html , a discssion on the Debian Users mailing list, has one person saying that "the Full Freedesktop Monty" for PID files "specif[ies] /run/user for that".

One can find many other examples.

So should my application be altered to use /var/run/user/$UID? Since it's a service (no active session) will pam_systemd clear/remove the directory? Or $XDG_RUNTIME_DIR? Is this something that is systemd-specific? Or is it a new standard across all Unices?

What is best practice nowadays, on systemd operating systems? Is it generally true for other (non-systemd, but Unix-like) operating systems as well?

Solution 1:

You ask:

Is /var/run/user/$UID the new /var/run for PID files?

The short answer is "no".

And the long answer still is "no".

Or to say it loud and clear: Traditional PID files must not go below /run/user/$UID/ (AKA /var/run/user/$UID/). Keep them in /run/ or /run/package/ as usual, as /run/user/$UID/ serves a completely different purpose for session services.

FYI:

• As noted by @Daniel B, /var/run today is a softlink which points to /run. This is not specific to systemd and can be found in most systems today. If you want to stay compatible to old systems, there are 2 solutions: Either stay with /var/run/ instead of /run/, or have somebody create a softlink for /run, pointing to /var/run on such old systems.

• Please also note that /run/user/$UID/ is specific to systems running systemd-logind. Also it is not supported on older systems.

To go into detail for those, who do not already know:

There are 3 types of services: System services, User services and Session services. All 3 types can either run in background or foreground, so this gives 6 variants.

Foreground system services traditionally were started by /etc/inittab, nowadays with systemd they are started via /etc/init/. Foreground system services usually do not need PID files, as they are controlled by init, and can be respawned if they fail.

Background system services traditionally were started by the runlevel scripts /etc/rc.d/. These usually need PID files, because they run in background, so there is no control about them running or not. PID files are very error prone (because there is no guarantee that the PID stays free after the service terminated) and are used to find the started services again when the rc-script needs to shut down the service. These PID files traditionally lived in /var/run and nowadays live in /run/package/ (or, if there is only one single file per package, in /run/package.pid).

User services are services which are started by a user, and need to live beyond the session of the user. For example, a Minecraft server, or something like a long living SAP query process which is triggered on-demand using a short-lived session. They are a bit different to system services, as they cannot use /run/. Instead they need to use /tmp/, or some directory below $HOME (which is problematic if $HOME is a network share, shared among several computers).

Foreground user services are sometimes a bit problematic, because they often need a tty, so if the user logs out they terminate. Hence there are numerous ways to let them live after the user has gone, like nohup or screen. But there can even be some exotic variants like socat tcp-connect:host:port exec:service.sh,pty.

Another variant of foreground user services are cron jobs which live in the crontab of the user. However, those cronjobs also can be background services, for example if they are not supposed to run in parallel, even if one process exceeds the time, when the next invocation by cron is done.

Background user services have the same problem as background system services, as they need to keep track which one already runs and control already running ones. In the past this has lead to various problems and fixes for things like directory traversal attacks and similar, due to the need to use /tmp where everybody is able to create files and directories.

However this has not changed yet, as /run/user/$UID/ is not meant for such user services. It was designed to solve a yet tougher problem with session services.

A session service is a service, which usually is started with the user logging in, and stopped with the user logging out. This sounds easy, but gains momentum if there is more than just one single session allowed per user. The difficult question to solve is: "when does a session really end"?

A session starts with the first login. That's easy to understand. But it does not necessarily end when this login logs out! So the session service can be started with the first login (or later), but quite often needs to continue to run, until the very last session of the user logs out.

Foreground session services are usually the easy variant. Like the X-Window dbus service which starts and terminates with the graphical login. But if you start to print a giant PDF, you certainly want this job either to complete successfully or being terminated cleanly without debris left behind, right?

These type of session services either need to continue to run in background, or there must be some way to clean up after the service was shot down if things die unexpectedly. And things die often in our mobile world. Think of your monitor. You can plug in and unplug a monitor any time. No need to reboot the computer. But what happens, if you unplug all screens? Well, X11 dies, of course. For some user service running in the X11 session this comes unexpectedly, perhaps in the middle of a longer running task.

This is where /run/user/$UID/ comes in handy, as this directory is wiped automatically after the last user session has finished. So services can trust that there will be a cleanup behind the user for everything stored in /run/user/$UID/!

So all session related services shall use this directory.

Also please note that we have dbus today. So you do not need to rely on PID files anymore just to find out if some service is running or not. This is especially true for session services, as there is something called "session", which allows you to handle things a bit better, like using shared memory segments or lock files which can live in /run/user/$UID/.

Things ain't easy. And to make it even a bit more complex there are things like screen, tmux or ssh (the one forking off the shell) which have 2 sides, one side is a user service (i. E. the daemon), the other side is a session service (the tty). While they are usually bound to a session, this need not be the case all times. For example, if you use ssh port forwarding and exit out of your shell, ssh stays open until the last forwarded port is closed. You can even open new ports as long as some other stays active. In case of such a "dual" service, something like a PID file might come in handy, and in this case it even might live under /run/user/$UID/.

(Note that cron could be used to open a user session, so probably /run/user/$UID/ is available for cronjobs, too. But if this is the case, /run/user/$UID/ becomes eligible for cleanup again after all cronjobs are finished. This, perhaps, means that files vanish which were handed over elsewhere, unlike when they live in /tmp/, as only a reboot is supposed to unconditionally remove files there.)

To sum it up:

If you have a properly designed service, you will never need plain PID files in /run/user/$UID/, because session services (the only one using this directory) usually have a quite better way (the session) to stay in control, even if they are running in background.

So if you see the need of a PID file for your service, it is very likely that you end up with something like /run/package/, /run/package.pid or /tmp/package-$UID/.

Only use /run/user/$UID/ if you want to make sure the file shall vanish as soon as the user logs off completely. Also note that the root user is not always logged in, so it is likely that there is no /run/user/0/.

And, please, never create the directories right below /run/user/ yourself!

Hopefully everything is so nice and clear now. But I have something to confess:

I lied to you. And I did this wittingly (but free from bad intent).

As a session (from systemd-logind's perspective) is not just bound to a login. Yes, something similar to login is involved, of course, but things are quite a bit more complex:

https://dvdhrm.wordpress.com/2013/08/24/session-management-on-linux/

However:

• For this posting here (the short, but wrong story) the answer to your question is "no".

• Luckily with the long (and true) backgrounds about sessions (see link), the answer still stays at "no".

• Only for some very few and very special cases for a very well defined purpose, you might consider putting your PID file into /run/user/$UID/.

Have a nice day.

Solution 2:

The systemd.exec manual says the following about its RuntimeDirectory option:

Takes a list of directory names. If set, one or more directories by the specified names will be created below /run (for system services) or below $XDG_RUNTIME_DIR (for user services) when the unit is started, and removed when the unit is stopped.

Additionally, on the pam_systemd man page , we can find the following information about $XDG_RUNTIME_DIR (emphasis mine):

Path to a user-private user-writable directory that is bound to the user login time on the machine. It is automatically created the first time a user logs in and removed on the user's final logout.

Using these two manuals, we can deduce that $XDG_RUNTIME_DIR (/run/user/$UID on my Arch install) is not the right choice for system services. Instead, they are usually put in /run (at least on Arch and Gentoo). It may also be feasible to create them in a RuntimeDirectory as described above.

/var/run is but a symlink to /run these days.