The remote root login was my first thought too. Especially when I saw the sentence "... I'm pretty careful with machine security ...".
What's bad about password auth if your password is strong enough?
What's bad about password auth if your password is strong enough?
This is very easy to do, and quite important for devices that can be stolen - including your desktop computer.
It's easy enough to setup ssh-agent on your desktop so you don't have to keep entering the password.