Changing the port to something other than the default (even to something obvious like 22222) will get rid of all the log spam. You could also use something like fail2ban, but changing the port is much simpler.
Just remember that the port argument to `scp` is -P and not -p like it is for `ssh`. :p