Why can't I run programs after closing SSH connection using setsid, nohup or disown? [duplicate]
I am attempting to run a program 'a.o' on a remote server. To keep it running after closing SSH connection, I tried the following:
setsid ./a.o
nohup ./a.o
./a.o &
disown -h %1
None of these options seem to work, and I cannot figure out why. Any ideas on how to fix this problem?
P.S. I am using a tunnel connection to connect to the server via a jump machine. Also, this is a program that involves CUDA code running on GPU. Do these have anything to do with the issue?
The nohup
Wikipedia Page has a reference to this issue specifically:
Note that nohupping backgrounded jobs is typically used to avoid terminating them when logging off from a remote SSH session. A different issue that often arises in this situation is that ssh is refusing to log off ("hangs"), since it refuses to lose any data from/to the background job(s).[6][7] This problem can also be overcome by redirecting all three I/O streams:
$ nohup ./myprogram > foo.out 2> foo.err < /dev/null &
Also note that a closing SSH session does not always send a HUP signal to dependent processes, such as when a pseudo-terminal has not been allocated.
As mentioned in comments, screen
and tmux
are perfect for this too. Personally, I prefer tmux
.