If ssh password out would be no problem, then why are so many APTs "wasting" their botnets with credentials stuffing?
Your assumption is wrong, and policies for key based auth eliminate the problem quite easily. Versus on the other hand: are you checking every colleague's password for length, charset, etc? All the time? On every server?
Probably not.
On SSH password auth: its secure if you use a long, random, not reused elsewhere password for every user. But it is also very easy to not do these things. SSH certs are just more convenient imo.
For ssh, the problem does not lie within password auth itself, but with weak passwords. A good password is more secure than a keypair on a machine whose files you can't keep private.