I don't know about anybody else, but in my mind I've always kind of thought of the term "haters" as short for "jealous haters".