How to check "apt-get upgrade" status after losing ssh connection?

The following logs are related to apt upgrades:

/var/log/apt/history.log
/var/log/apt/term.log
/var/log/dpkg.log

If the command was dist-upgrade, there are additional logs in:

/var/log/dist-upgrade

FYI, it is usually safe to just re-run the upgrade and apt will continue where it left off when the process died due to disconnection. However...

A GNU Screen Primer:

When ssh'ing into a remote server and starting a long-running process in the foreground, it is best practice to use GNU Screen. Screen provides a virtual terminal that continues running even if your ssh connection is lost.

Install screen:

sudo apt-get install screen

Run screen:

screen

After running screen you will get a command line prompt as with a normal terminal. You can then run the upgrade from inside screen:

sudo apt-get upgrade

To understand how this works, "detach" screen by pressing Ctrl+a, d. This will return you to the non-screen terminal. You can see the list of running screens with

screen -list

If you only have one screen running, you can reattach it with:

screen -raAd

(This detaches screen in case it is attached elsewhere, and reattaches it to the terminal you are currently running.)

Typically you cannot scroll 'normally' from within screen without some extra setup. To scroll within screen, press Ctrl-Esc to enter cursor mode. You can then scroll down and up with j and k. Press Esc again to exit cursor mode.

There are many more resources on the net available for additional screen functions. It is an invaluable standard tool for system administration.

See also:

  • How to keep processes running after ending ssh session?

To see real-time output from background apt job, use:

sudo tail -f /var/log/apt/term.log

In addition to doublerebel's answer, I noticed an alternative today.

I went to bed last night after starting an upgrade over SSH. I stupidly forgot to start it in screen and lost my SSH session overnight.

I was just about to start researching retty when I noticed that root had started a screen session.

me@GAMMA:~$ ps aux | grep -E 'release|upgrade|apt'
root      6208  0.0  0.0  29140  1628 ?        Ss   01:57   0:05 SCREEN -e \0\0 -L -c screenrc -S ubuntu-release-upgrade-screen-window /tmp/ubuntu-release-upgrader-1h6_g4/raring --mode=server --frontend=DistUpgradeViewText
root      6209  0.2  5.6 287428 93144 pts/2    Ss+  01:57   3:13 /usr/bin/python /tmp/ubuntu-release-upgrader-1h6_g4/raring --mode=server --frontend=DistUpgradeViewText
root      6239  0.0  0.0  50052  1184 ?        Ss   01:58   0:00 /usr/sbin/sshd -o PidFile=/var/run/release-upgrader-sshd.pid -p 1022
root      7306  0.0  4.6 287432 77284 pts/2    S+   02:43   0:08 /usr/bin/python /tmp/ubuntu-release-upgrader-1h6_g4/raring --mode=server --frontend=DistUpgradeViewText
me       26829  0.0  0.0   9440   956 pts/5    S+   22:18   0:00 grep --color=auto -E release|upgrade|apt

So I listed root's screens and attached to it:

me@GAMMA:~$ sudo screen -list
There is a screen on:
        6208.ubuntu-release-upgrade-screen-window       (12/11/2013 01:57:58 AM)        (Detached)
1 Socket in /var/run/screen/S-root.
me@GAMMA:~$ sudo screen -x -r

And Bam! I was back in the game.


I had an upgrade in progress (asking me whether to keep or replace a config file; this information I took from /var/log/apt/term.log) when my terminal was closed by another person. Restarting apt upgrade complained about a lock file being held.

My fix was to do the following:

sudo killall apt # to release the lock
sudo dpkg --configure -a # to fix the interrupted upgrade
sudo apt upgrade # to make sure no packages are left not upgraded