Sure, but have you heard of reducing the attack surface? If you need to have to be able to log in at all times then youre probably at a scale that you have oncall processes and multiple people that can respond to incidents at a moments notice and having pub key auth enabled only makes sense. If you dont need that then youre probably small enough that that enablig only public key auth or putting it behind a vpn suffices. And having something like wireguard is much better than having something like password login enabled.

Anyone who sacrifices security for convenience is asking for trouble.