1. Disable password authentication altogether
2. Create and use keypairs
3. Add an "AllowUsers foo" line to /etc/ssh/sshd_config so that only your user is allowed to SSH
4. Install sshguard or fail2ban and set the ban limit to a week
There are other things you can do, but I find this to be the lowest hanging fruit for the best security.