Yeah, running SIP on a standard port without some serious firewall based rate limiting for unknown traffic is almost impossible.
I tried running a PBX on UDP 5060 and got >4GiB of logged register attempts in a few hours after opening the port, while asterisk was running at 100% CPU just rejecting the registration attempts the whole time.
It's insane compared to any other public service I run.