Working around the stale pidfile problem after hard restart kills my daemon

Use daemontools and see Process Management.

Thank you for the hints @Dennis and @coredump.

I found out some additional information that helped me unravel the mystery.

1. I wondered why every other daemon recovered fine. It turns out there is code in /etc/rc.d/rc.sysinit to clean up all pidfiles in /var/run and /var/lock at boot.

2. I had configured my daemon to put its pidfile elsewhere because of trouble with SELinux preventing me from "using potentially mislabeled files".

So I haven't fixed it yet due to the SELinux issues, but the answer I think is "put your pidfile in /var/run or /var/lock and it will work next time"

The script is the same, the startup process just executes the 'start' action on the sysvinit scripts.

Why don't you check if the pid on the pid file is right, and if don't delete it and create a new one with the right pid?

EDIT: You can grep ps with the pidfile to see if the process still exists. Or do the other way around. Check RedHat initscripts, I am sure they have some helper functions to do that, like pidofproc.