If all you're worried about is keeping the low-level noise out of your logs and discouraging waste of resources on brute-force attacks (which won't work because of course password auth is disabled), I think it's sufficient to use the built-in iptables rate-limit feature on SYN packets.

UFW has a feature that will implement this with a limit of 6 SYN packets in 30 seconds from individual IPs.

    ufw limit ssh/tcp

This may not be ideal for some services like http without persistent connections, and if fail2ban provides extra exploit protection it could be worthwhile.