Though port scanning can be (and maybe even frequently is) done with malicious intent by looking for misconfigured/bugged servers, I disagree that it's inherently malicious. Port scanning is just about checking to see what services a host is offering you. It's like going to a random shop at a mall and asking what services they provide. Would asking about their services be malicious?
It feels like the reason asking about services is considered malicious is because shops frequently give out info to the public that they shouldn't have. It's like:
client: What services do you provide?
shop owner: Well, I can provide you with a list of all my clients along with their personal information they entrusted to me.
So, is the client being malicious for asking or is the shop owner the one that was in the wrong for mistakenly providing that info to the public?
I feel the only reason we don't blame the shop owner is because even though he's the one that mistakenly discloses private info, sometimes he's just following a script written by a random programmer unassociated with him. Maybe the response was a mistake on the programmers part, maybe it was a mistake in how the shop owner used the script (a configuration error). In the end, it's simpler to blame the client for asking out-of-the-box questions (after all, most clients just come in to ask if you're giving out flyers/pamphlets because that's what everybody does) and so they don't feel responsible for the response that results.
I can provide a shop that also offers things different than http(s) with open access to the public. It shouldn't be a crime/violation to ask me if I offer them.