Do background processes get a SIGHUP when logging off?
This is a followup to this question.
I've run some more tests; looks like it really doesn't matter if this is done at the physical console or via SSH, neither does this happen only with SCP; I also tested it with cat /dev/zero > /dev/null. The behaviour is exactly the same:
- Start a process in the background using
&(or put it in background after it's started usingCTRL-Zandbg); this is done without usingnohup. - Log off.
- Log on again.
- The process is still there, running happily, and is now a direct child of
init.
I can confirm both SCP and CAT quits immediately if sent a SIGHUP; I tested this using kill -HUP.
So, it really looks like SIGHUP is not sent upon logoff, at least to background processes (can't test with a foreground one for obvious reasons).
This happened to me initially with the service console of VMware ESX 3.5 (which is based on RedHat), but I was able to replicate it exactly on CentOS 5.4.
The question is, again: shouldn't a SIGHUP be sent to processes, even if they're running in background, upon logging off? Why is this not happening?
Edit
I checked with strace, as per Kyle's answer.
As I was expecting, the process doesn't get any signal when logging off from the shell where it was launched. This happens both when using the server's console and via SSH.
Answer found.
For BASH, this depends on the huponexit shell option, which can be viewed and/or set using the built-in shopt command.
Looks like this options is off by default, at least on RedHat-based systems.
More info on the BASH man page:
The shell exits by default upon receipt of a SIGHUP. Before exiting, an interactive shell resends the SIGHUP to all jobs, running or stopped. Stopped jobs are sent SIGCONT to ensure that they receive the SIGHUP. To prevent the shell from sending the signal to a particular job, it should be removed from the jobs table with the disown builtin (see SHELL BUILTIN COMMANDS below) or marked to not receive SIGHUP using disown -h.
If the huponexit shell option has been set with shopt, bash sends a SIGHUP to all jobs when an interactive login shell exits.
It will be sent SIGHUP in my tests:
Shell1:
[kbrandt@kbrandt-opadmin: ~] ssh localhost
[kbrandt@kbrandt-opadmin: ~] perl -e sleep &
[1] 1121
[kbrandt@kbrandt-opadmin: ~] ps
PID TTY TIME CMD
1034 pts/46 00:00:00 zsh
1121 pts/46 00:00:00 perl
1123 pts/46 00:00:00 ps
Shell2:
strace -e trace=signal -p1121
Shell1 Again:
[kbrandt@kbrandt-opadmin: ~] exit
zsh: you have running jobs.
[kbrandt@kbrandt-opadmin: ~] exit
zsh: warning: 1 jobs SIGHUPed
Connection to localhost closed.
Shell2 Again:
strace -e trace=signal -p1121
Process 1121 attached - interrupt to quit
pause() = ? ERESTARTNOHAND (To be restarted)
--- SIGHUP (Hangup) @ 0 (0) ---
Process 1121 detached
Why does it still run?:
Advanced Programing in the Unix Environment by Stevens covers this under section 9.10: Orphaned Process Groups. The most relevant section being:
Since the process group is orphaned when the parent terminates, POSIX.1 requires that every process in the newly orphaned process group that is stopped (as our child is) be sent the hang-up signal (SIGHUP) followed by the continue signal (SIGCONT).
This causes the child to be continued, after processing the hang-up signal. The default action for the hang-up signal is to terminate the process, so we have to provide a signal handler to catch the signal. We therefore expect the printf in the sig_hup function to appear before the printf in the pr_ids function.
I ran some tests using CentOS 7.1 and bash. Note this means huponexit is off by default, and was off for the majority of my tests.
You need nohup when you start a job in a terminal, because if you close that terminal without exiting the shell cleanly, the terminal sends bash the SIGHUP signal to the shell, which then sends it to all children. If you exit the shell cleanly- meaning the job must already be in the background so you can type exit or hit Control-D at the command prompt- no signals of any sort are sent to the background job from bash.
Test:
Terminal 1
$ echo $$
16779
Terminal 2
$ strace -e signal -p16779
Process 16779 attached
(close terminal 1, seen in terminal 2):
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=16777, si_uid=3000090} ---
rt_sigprocmask(SIG_BLOCK, [CHLD TSTP TTIN TTOU], [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_SETMASK, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], 8) = 0
rt_sigprocmask(SIG_SETMASK, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], NULL, 8) = 0
rt_sigaction(SIGHUP, {SIG_DFL, [], SA_RESTORER, 0x7f7ace3d9a00}, {0x456880, [HUP INT ILL TRAP ABRT BUS FPE USR1 SEGV USR2 PIPE ALRM TERM XCPU XFSZ VTALRM SYS], SA_RESTORER, 0x7f7ace3d9a00}, 8) = 0
kill(16779, SIGHUP) = 0
rt_sigreturn() = -1 EINTR (Interrupted system call)
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=16779, si_uid=3000090} ---
+++ killed by SIGHUP +++
Job doit.sh:
#!/bin/bash
imhupped() {
echo "HUP" >> /tmp/outfile
}
trap imhupped SIGHUP
for i in $(seq 1 6); do echo out $i >> /tmp/outfile; sleep 5; done
Start it in the background in Terminal 1:
Terminal 1
$ ./doit.sh &
[1] 22954
Strace it in Terminal 2; close Terminal 1 after a couple of loops:
Terminal 2
$ strace -e signal -p22954
Process 22954 attached
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=22980, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn() = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f7a5d547a00}, {0x43e4b0, [], SA_RESTORER, 0x7f7a5d547a00}, 8) = 0
...
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=21685, si_uid=3000090} ---
rt_sigreturn() = -1 EINTR (Interrupted system call)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=23017, si_status=SIGHUP, si_utime=0, si_stime=0} ---
rt_sigreturn() = 0
...
Output in Terminal 3:
Terminal 3
out 1
out 2
out 3
HUP
out 4
out 5
out 6
However, if you exit bash, it simply exits without sending any signal to the child at all. The terminal will exit because it no longer has a child, but of course there is no one to HUP because the child shell is already gone. The SIGINT, SIG_BLOCK and SIG_SETMASK you see below are due to the sleep in the shell.
Terminal 1
$ ./doit.sh &
26275
Terminal 2
$ strace -e signal -p26275
Process 26275 attached
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26280, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn() = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
(..."exit" is typed in bash, notice no new signals sent...)
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26303, si_status=0, si_utime=0, si_stime=0} ---
rt_sigreturn() = 0
rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x43e4b0, [], SA_RESTORER, 0x7f5edd3a5a00}, {SIG_DFL, [], SA_RESTORER, 0x7f5edd3a5a00}, 8) = 0
Terminal 3, output
out 1
out 2
out 3
out 4
out 5
out 6
Interestingly, I set huponexit to be on with shopt -s huponexit; shopt (the latter shopt to review), then performed the last test, and again bash sent no signal to the background process. Even MORE interstingly, as we have seen bash did send the signal to the background process after it received it from a terminal that closed in its face. It seems as though huponexit had no bearing one way or the other.
I hope this removes any mystery or confusion regarding at least bash's huppiness, about when and how the HUP signal is sent. At least my tests were completely reproducible, for me. I would be interested to know if there are any other settings that may be affecting bash's behavior.
And, as always, YSMV (Your Shell May Vary).
Addendum 1
When I run a shell as exec /bin/sh, then run the script as /bin/sh ./doit.sh &, then exit the shell cleanly, no signals are sent to the background job and it continues to run to completion.
Addendum 2
When I run a shell as exec /bin/csh, then run the script as /bin/sh ./doit.sh &, then exit the shell cleanly, no signals are sent to the background job and it continues to run to completion.