The term turned negative because it represented something that no one wanted to be associated with. Even if you were to use the phrase "unicorn-blessed" in the same context, over time, it could also develop a negative connotation. It's not the terminology that's a problem, but the situation it describes.
"Idiot", "moron", and "imbecile" used to be medical terms.