Have you tried/are you interested in setting a different port number on the server? Setting a different port number and using ssh_config on your clients (so you don't have to keep specifying the different port) helps a ton. I've done this with a Google 2FA solution[1] and honestly haven't seen a single rogue attempt.
----- [1] https://www.digitalocean.com/community/tutorials/how-to-set-...