Sysadmins with root access should be able to handle a random 8 character or longer password, and that number is large enough for a secure public accessible ssh. If your not a sysadmin or unable to remember a random password, try go with a passphrase like "correct horse battery staple". If you have too many machines and thus can't remember passwords, then use a master password locally with a bit larger password and store certs there.
To do some math to illustrate the security of a random 8 character long password, someone would need to fill a 100/100 line for several hundred years to get through them all. By that time you should have noticed the constantly full connection, and be happy you haven't died of old age yet even after your 200th birthday.
However, 8 character passwords are not the suggested length by security experts. They suggest using a 10 character long password, as that is also secure in the case that your password hash somehow got lost.
For the several servers that I have, I have been more worried about the logs from failed attempts than I have been of anyone guessing the password. Logs wears on the hard drive, so one might want to install fail2ban to lower the number of writes to it. It also decrease the noise level in the server room.