Nothing wrong with Bitcoin itself but the rampant speculation is what draws the proponents and critics alike.
If Bitcoin just worked like a peer-to-peer neutral currency and had no speculative element, most people would have no issue with it.
The annoying element is when it is used as a get-rich quick pyramid scheme.