Absolutely. But it's definitely a double standard based on associations. It's the whole freedom fighter/terrorist coin argument. If you're against the status quo, you're a cyber terrorist who deserves no trial and bullet to the brain. If you're defending the status quo you're a security specialist and you're the beacon of light in a dark world.
So if you are a "hacker", this won't define what people think of you necessarily. It depends on what you do with that power.
Things you should not do, unless you do want to be labeled a cyber terrorist:
* Changing the parameters of a companies URL that might compromise security, then posting that information to the public.
* Downloading copyright protected material.
* Creating porn sites.
* Posting extremist/terrorist/anarchist ideas on the internet.
* Creating a shortcut or bypass that cuts into the profits of a monopolistic company.
If you avoid these things, and hide the fact that you understand computers at an intermediate level, you should be fine.