Some will proudly wear the badge and call themselves hackers because of the above. Facebook is huge on the term itself, even getting themselves the vanity address 1 Hacker Way.
Unfortunately, a huge chunk of the world does in fact view the word hacker very negatively like your manager and even in some reports I've come across, the authorities you don't want messing with you. For these reasons, I try to stay away from ever calling myself a hacker of any kind. It's a term. It can mean whatever you claim it to but it won't matter if others don't see it that way, it causes confusion, or at the end of the day means nothing to anyone else but you. After all, the only person that cares is the person claiming the title.
Having deep knowledge at something and being able to do what you want with it, makes you a hacker.
Note that sometimes there is a confusion with hackers and script kiddies. Script kiddies in short are those with little or no real knowledge use instructions and tools causing damage only.
Hope this helped.
The important thing is the effect/intent is completely independent of the definition. People inject their intentions into the definition, but on it's own, hacking is all about curiosity.