Keep running a python program even after logging-off the ssh session [duplicate]

I python script called scrape.py. As the name suggests it is a web crawler and I want it to run continuously on my Linux machine even after I log out and end the ssh session. I read a lot around here and there were many suggestions of using nohup python scrape.py &. This worked well but the script execution stopped as soon as I logged out. I also explored the supervisor application but I don't have root permissions so it fails too. Can you suggest me a way to solve this problem? Thank you.

Solution 1:

There are several approaches (tools and commands) that you can use for similar purpose:

1. Use terminal emulators as tmux and screen:

   • tmux - terminal multiplexer: it enables a number of terminals to be created, accessed, and controlled from a single screen. tmux may be detached from a screen and continue running in the background, then later reattached. Examples of its basic usage are provided here and here.

   • screen - screen manager with VT100/ANSI terminal emulation. It is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells)... Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the user's terminal. Example of usage is provided here.

2. Use detached commands executed in the background (read also):

   • nohup - run a command immune to hangups, with output to a non-tty; Run COMMAND, ignoring hangup signals. The common syntax to push a command into the background and detach it from the current shell session: nohup COMMAND & or nohup COMMAND >/dev/null 2>&1 & if you want to suspend the info-file created by nohup. Examples of its basic usage are provided here and here. Also here is shown how to use it via ssh command.

   • disown - similar to nohup but it is shell (zsh, ksh, bash) builtin; used to remove jobs from the job table, or to mark jobs so that a SIGHUP signal is not sent to them if the parent shell receives it (e.g. if the user logs out). Example of usage is provided here.

   • setsid - creates a session and sets the process group ID. Sample usage setsid "COMMAND". More examples here.

3. Use the huponexit Bash Option:

   $ ./script.py > output.txt &
   [1] 7461
   $ shopt -s huponexit
   $ exit
   

   Reference:

   • How To Use Bash's Job Control to Manage Foreground and Background Processes at DigitalOcean

4. Use schedule managers as at and crontab:

   • at - execute commands at a later time; at and batch read commands from standard input or a specified file which are to be executed at a later time, using /bin/sh. Sample usage: echo "COMMAND" | at now + 1 minute. More examples here.

   • crontab - schedule periodic background work; it is well known for other purposes, but you can specific certain time moment when your command will be executed only once. A nice crontab calculator is provided here.

5. Create system service and start it, see:

   • How do I run a single command at startup using systemd?
   • How to run scripts on start up?