What's even more annoying is that -P for scp has to be put before the target while -p for the ssh client I am using can be put after the target.

So when you write your scp command and then realize you forgot to put in the port you have to go back and put it before the target.