How to ban an IP temporarily after several incorrect password attempts?

Solution 1:

I recommend looking at fail2ban to do this for you:

https://help.ubuntu.com/community/Fail2ban

(Also, yes, I would just do a fresh install, especially if they got root access.)

Solution 2:

fail2ban is a good solution, but I'm a fan of DenyHosts, which is available in the repos. Just do sudo apt-get install denyhosts, and that will install DenyHosts and start it with a pretty sensible configuration.