They are a hazard because drivers need to take actions to avoid them. It turns the road into an obstacle course, except with unpredictable obstacles that rarely follow traffic rules like red lights.
I do see this remark ("you dumb kids on your bikes break all the traffic laws!!") on all the web forums for the local newspaper, whenever any article comes along about bike lanes or whatever. I just don't see people on bikes breaking traffic laws anywhere near as often as people in cars.
As for needing to avoid things, would you call traffic circles or pedestrians a "hazard" as well? Again, you're in a two-ton metal box, none of this stuff is going to do anything more than scratch your paint.