So standardize on a non-standard port, document it and when you do have ssh probes assume it's targeted rather than just another l33t child running du-jour crack(x).

Moving to non-standard means more sophisticated discovery and possibly a more sophisticated attempt.