They aren't doing anything funny, nor are they asking you for permission to do anything funny.
They scan your ports. To the unobservant, that can look like they are trying to crack your systems. What is wrong with recognizing their IP (adding it to your "whitelist"), and knowing that it probably isn't something worth investigating.