How to speed up SSH login?

From outside of my house, whenever I login to my Ubuntu server using SSH, it takes about 6 seconds for me to get the prompt for password, however when I login to my web hosting server it takes about 1 second. What can I do to speed this up?

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.04
DISTRIB_CODENAME=hardy
DISTRIB_DESCRIPTION="Ubuntu 8.04.1"
$ ssh -v
OpenSSH_4.7p1 Debian-8ubuntu1.2, OpenSSL 0.9.8g 19 Oct 2007
$ cat /proc/cpuinfo
model name      : Dual-Core AMD Opteron(tm) Processor 1210
cpu MHz         : 1000.000
cache size      : 1024 KB
$ cat /proc/meminfo
MemTotal:      2074528 kB

The number one reason I've seen for this is a configuration option in SSHD UseDNS this option (enabled by default) causes the server to perform DNS resolution on the incoming requests. A time consuming operation. I've seen logins go from one minute plus waiting for password prompt to under a few seconds. If you edit /etc/ssh/sshd_config on the server and add (if it's not there) at the bottom UseDNS no then restart the SSH daemon with service ssh restart you should see an improvement next time you connect.

For me, the reason number 2 (after the server-side UseDNS option) for long SSH session logins are client side attempts to connect using IPv6 (which, obviously, isn't set up correctly on my network - or almost any other network, for that matter).

See HOWTO: Speed up SSH login on Ubuntu forums.

The "solution" is to enable IPv4 only:

1) either for the given SSH client invocation:

ssh -4 login@hostname

2) or globally in ssh client conffiguration in /etc/ssh/ssh_config:

Host *
   AddressFamily inet

Of course, it would be more correct to set up IPv6 on your network properly, but who has the time for that :)

Try adding the next option with your ssh command:

-o "PreferredAuthentications=password"

(this prevents ssh from negotiating any other authentication method, and speeds up the password promtp!)

Additionally, type this on the remote machine (as the user you would log in as) to suppress any MOTD messages:

touch ~/.hushlogin

Doesn't make as much a difference as turning off UseDNS but it might help on slower connections.

As you are using a rather old Ubuntu version, it might well be this bug: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/300151

Restarting dbus (/etc/init.d/dbus restart) might help.