if you want to get fancy and/or over engineered you would use systemd templated units to setup sshd@.service and a ListenAddress in the config listening on %i. Then you could bring up sshd@(expectedip).service for each expected IP

.. but that doesn't gain all that much tbh.if anything the only hesitation I'd have on listening to * and relying on firewall rules is if the service comes up before its configured. but exposing sshd isn't even that bad