How can I prevent myself from getting logged out automatically in a multi-user setup?

You can check that you have enough TTY's:

Open a terminal and do:

grep tty /etc/default/console-setup

It should show (check the last number, it should say 6):

ACTIVE_CONSOLES="/dev/tty[1-6]"

If not, edit the /etc/default/console-setup file by typing:

gksudo gedit /etc/default/console-setup

And look for ACTIVE_CONSOLES, and make the line look this:

ACTIVE_CONSOLES="/dev/tty[1-6]"

In the end I lived with this for a while and ended up upgrading Ubuntu to a newer version and it went away. So not sure what it was but probably a bug somewhere..