While in principle that is possible, in practice almost all home routers are based on Linux, and Linux netfilter NAT implementation distinguish connections based on port and IP, not just port, so this would not work.
The poke a hole to outside world to a random server, log the port allocated to you by your router and have someone else use this to connect to you is the basis of STUN protocol.
BT, one of the largest ISP's in the UK, only allow the configuration of destination IP and external/internal ports[0].
I've never expected my NAT to do anything other than map ports. I can see why the ability to map source IPs to different ports would be useful but relying on that as a security feature feels like a foot-gun. I wouldn't feel comfortable exposing an application that doesn't have some form of authentication and/or blacklisting.
[0] https://portforward.com/bt/home-hub-6/Port%20Forwarding.jpg