I have certain hosts behind a single IP and forward SSH to them on arbitrarily chosen ports.
Do I still get random logins on those ports?
Why yes I do.
Does putting SSH on a different port make any difference?
No it doesn't.
Putting SSH on a different port is either done for a specific reason, or you're just deluding yourself that you've somehow reduced your attack surface.