If every network connection your system makes authenticates the client to the server, authenticates the server to the client, and encrypts the dataflows then you don't have to worry if your routers are hacked.

A typical way to do this is to use TLS, verify certificates, and have a password for the client.

It's a well known & age old principle in building networked systems.

https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...

(But of course these fallacies are so well known exactly because programmers have proven their inability to reliably internalize them...)