For a home user, you can also set SSH to be Local LAN only, which is how I have mine set anyway.

My point was that if the attackers cared enough to put (not much) effort into keeping control of these routers then neither of those approaches is likely to be sufficient.

This sort of thing is why there is such a emphasis on TPM and trusted boot on modern PCs.