systemctl shows inactive / dead, but service is running
I would like to manage a TeamSpeak 3 server with systemd (systemctl).
When I start / stop / restart the TeamSpeak 3 server using systemctl command, systemctl is returning the correct status:
$ systemctl status ts3server.service
● ts3server.service - TeamSpeak 3 Server
Loaded: loaded (/etc/systemd/system/ts3server.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-02-12 23:15:18 CET; 1min 11s ago
Main PID: 842 (ts3server)
CGroup: /system.slice/ts3server.service
└─842 ./ts3server inifile=ts3server.ini
Feb 12 23:15:18 hostname.local systemd[1]: Starting TeamSpeak 3 Server...
Feb 12 23:15:18 hostname.local ts3server_startscript.sh[838]: Starting the TeamSpeak 3 server
Feb 12 23:15:18 hostname.local ts3server_startscript.sh[838]: TeamSpeak 3 server started, for details please view the log file
Feb 12 23:15:18 hostname.local systemd[1]: Started TeamSpeak 3 Server.
But when the service is stopped otherwise (eg. through an automated update process) and then started again - without systemctl - the systemctl status command returns an incorrect status:
$ systemctl status ts3server.service
● ts3server.service - TeamSpeak 3 Server
Loaded: loaded (/etc/systemd/system/ts3server.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Mon 2019-02-11 03:15:16 CET; 1 day 19h ago
Condition: start condition failed at Mon 2019-02-11 03:15:16 CET; 1 day 19h ago
└─ ConditionPathExists=!/home/teamspeak/.update.lock was not met
Process: 21740 ExecStop=/home/teamspeak/ts3server_startscript.sh stop (code=exited, status=0/SUCCESS)
Process: 29220 ExecStart=/home/teamspeak/ts3server_startscript.sh start (code=exited, status=0/SUCCESS)
Main PID: 29224 (code=exited, status=0/SUCCESS)
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
The PID file exists in this moment:
$ stat /home/teamspeak/ts3server.pid
File: /home/teamspeak/ts3server.pid
Size: 4 Blocks: 8 EA Block: 4096 regular file
Device: ca01h/51713d Inode: 534471 Symbolic Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1001/teamspeak) Gid: ( 1001/teamspeak)
Access : 2019-02-12 23:15:09.192927211 +0100
Modified: 2019-02-12 23:15:09.188927218 +0100
Changed : 2019-02-12 23:15:09.188927218 +0100
Birth : -
The process with this PID is running and I can connect to this TeamSpeak 3 server.
/home/teamspeak/.update.lock exists while the automated update process is running. After the update, the file is removed / deleted and does NOT exist anymore.
systemctl shows the correct status, when I kill the running service PID as systemctl restarts the service using systemd correct later.
Here is my /etc/systemd/system/ts3server.service file:
[Unit]
Description=TeamSpeak 3 Server
After=network.target mysqld.service
ConditionPathExists=!/home/teamspeak/.update.lock
[Install]
WantedBy=multi-user.target
Alias=ts3server.service
[Service]
User=teamspeak
Group=teamspeak
WorkingDirectory=/home/teamspeak/
ExecStart=/home/teamspeak/ts3server_startscript.sh start
ExecStop=/home/teamspeak/ts3server_startscript.sh stop
ExecReload=/home/teamspeak/ts3server_startscript.sh restart
PIDFile=/home/teamspeak/ts3server.pid
Restart=always
Type=forking
Why can systemctl not return the correct status, although the TeamSpeak 3 server is started correct on a different way?
Solution 1:
You should change the Type to simple in your unit file and it will work. IIRC forking requires an explicit call of fork() method, while you start it from a bash script and there simply is no fork() call.
Solution 2:
The answer to your "why" question is that systemd is looking for the PID that it knows of (842 in your example output above), it does not track the service's own PID information. If that process is no longer running, systemd has no idea what's going on with the process. Nor can it restart it (since the ports would already be bound to by the new instance).
Probably a cleaner way to do the upgrade is
- Use systemctl to stop the service
- Perform the update, and then
- Use systemctl start the process.