SSH - sent password / access granted very slow

This is typically caused by slow DNS lookups. Add this to your sshd_config:

UseDNS no