Still worthwhile to firewall/filter ssh access except from specific IPs?
Solution 1:
I'd say it's still worth avoiding a large attack surface in the form of your SSH server.
Some things I do (varying between different machines) are:
- iptable level connection rate limiting
- run SSH on a different port
- fail2ban
- opie
- no root logins
- ssh keys only
Running ssh on a different port avoids a lot of the bots and scans, but is a little bit more work when connecting (but an entry in ~/.ssh/config helps). It won't stop a determined hacker, just bots etc
Something like fail2ban + iptables, or iptables connection rate limiting is likely worth doing. It's quite simple to set up, but will quickly slow down anyone repeatedly trying things, and shouldn't affect you.
No root logins / OPIE / ssh keys only should help for most, but not all zero days, and will help with lots of other situations too. It can make the setup of a new machine / new admin a bit longer, but what it offers should be worth it.
Solution 2:
While there may not be (many) kiddies out there with those kind of exploits going on -- there are DEFINATELY zombie boxes with scripts that do it for them.
That being said -- there are still nasty hacks that can happen because of pin-holed IP addresses -- your security is only as strong as the remote machine/ip's security.
Look into Tunneling / VPN technologies at the very minimum. Also throw away passwords (authenticators and the like) if your really concerned about security.