Also when securing a box with public key encryption you should be configuring sshd to disable password authentication and disable root login. Editing an extra line in the config file isn't going to throw your schedule.
Security, like everything else out there, should be prioritized according to ROI. This is a pretty good ROI...maybe not better than picking good passwords, but definitely better than many practices that IT departments advocate.
Personally, this is why I change SSH ports every time on a public service and add extra firewall rules if possible. If for some reason I want to watch port 22 “attacks”, I can do so.
I’m not even sure I place this in the security OR obscurity categories at this point ... more of a disk hygiene issue.