Depending on the situation, it could be.
While not an issue any longer (well, at least for those without data caps -- man do I hate those!), ~30 years ago I worked for a network equipment manufacturer as an integrator/tester/3rd level support guy (yes, it was a small company) and was testing our IP stack.
I got the (not so) bright idea of running continuous pings to random IP addresses (changing them every couple days) to verify stability and identify possible memory leaks.
One of the addresses was someone in Australia who was charged by the byte (or packet...it was a long time ago) and after a day or two, we received some very angry telephone calls from them. Oops.
This person (and rightly so) felt I was abusing their internet link, even though I didn't know or care about them or the contents of their network.
Like I said, this isn't (or at least not for the most part) an issue any more, nor is it something more intrusive than port scanning, but it points up the idea that "abuse" is not a black and white thing.
Run port scans against my IP addresses and the absolute worst that might happen is me scanning you right back (yes, I know, that sort of thing is generally frowned upon. So sue me -- nobody has yet).
Run those same scans against government/military sites and you may well soon have a knock (or a battering ram) on your door.
Context matters.