To make matters worse, the debian openssh .deb, that i installed to get the client, configured an autostart for sshd, without asking.
It's possible and fairly straightforward to disable autostart of services through either old-school SysV init or, I suspect, systemd.
Upshot: running complex systems isn't entirely trivial.
But regardless, starting sshd on installation shouldn't matter because proper passwords and firewall should be the first step.
Due diligence on my part was lacking, agreed.