Why are the contents of /dev/shm/ is being removed automatically

On a RHEL 7.2 machine, if I create a POSIX shared memory in a console session, then query its presence in /dev/shm from an SSH session, it shows the shared memory file during the first time, but after that it gets deleted mysteriously.

Finally I've broken down the test case to the following steps:

1. On box1, do touch /dev/shm/sample
2. tailf /dev/shm/sample on box1. It will be accessible.

3. On box2, do ssh user@box1 "ls -l /dev/shm/"

   -rw------- 1 user user        1 Aug 25 17:12 sample
   

4. Do step 3 again, and this time I don't see the file.

5. On box1, tailf shows that the file has been deleted.

   tail: '/dev/shm/sample' has become inaccessible: No such file or directory
   

I've observed is that all the files corresponding to that particular user are getting deleted from /dev/shm, even if that is a tree of directories, containing files.

I've tried to monitor the file, strace on sshd, etc.

I have tried auditd with the following rules, but no luck:

## This file is automatically generated from /etc/audit/rules.d
-D
-b 1024
# monitor unlink() and rmdir() system calls.
-a exit,always -S unlink -S rmdir

Can someone explain what is going wrong here?

Solution 1:

From my answer:

After hours of searching and reading, I found the culprit. It's a setting for systemd. The /etc/systemd/logind.conf contains default configuration options, with each of them commented out. The RemoveIPC option is set to yes by default. That option tells systemd to clean up interprocess communication (IPC) for "user accounts" who aren't logged in. This does not affect "system accounts"

In my case, the files and directories were being created for a user account, not a system account.

There are two possible solutions:

1. Create the files with/for a system user -- a user created with the system option (adduser -r or adduser --system)
2. Edit /etc/systemd/logind.conf, uncomment the line RemoveIPC=yes, change it to RemoveIPC=no, save, and reboot the system

In my case, I went with option #2 because the user was already created.

References:

• RemoveIPC prematurely removes IPC segments
• Change default value of RemoveIPC in logind.conf
• PostgreSQL: Systemd
• Oracle: 3.5 Database Installation and Operation Fails if RemoveIPC=yes Is Configured for systemd

Solution 2:

/dev/shm provides a view of the shared memory that appears like a file system. There are system calls to create, use, and delete shared memory segments. Shared memory is intended to be used by co-operating programs to allow access to shared data structures. Depending on the mode they were created with, shared memory segments may be removed when no processes are using them. This prevents shared memory from being lost if the programs using them crash or otherwise exit without cleaning up.

If you have a kernel that mounts /dev/shm, then it should be listed as such in /etc/mtab. The permissions should be drwxrwxrwxt which prevents other users other than root from removing the files. If files for a particular user are being removed from /dev/shm it will be either that user or root that is removing them. Check for: processes running as the user; cronjobs running as the user; or scripts run a during login/logout.

Try logging in several times without logging out and checking for the file. If the file sticks around, then it is unlikely a process running as the user or a crontab entry. If it is deleted on the first logout, then it is likely a cleanup script run on logout.

If you want to create a file system in memory, there is the tmpfs file system. This is commonly used for /tmp, /var/run and other file systems that should be empty following a reboot. Files in tmpfs may be paged out to the swap files if memory is used for other purposes.