This is a lot of complexity compared to just not having ssh open to the world (on whatever port you choose to use).

Restrict it to the networks where authorized users will be connecting.