My own experience is the exact opposite. Out of all the times in my life I can recall ever having heard the word "retarded" used, I cannot think of any reason to suspect that any of them were meant as anything other than a synonym for "idiotic".

Which, of course, also referred to clinical mental disability at some point in history. As did "moronic", "imbecilic" and others. But nowadays they're really all just strong forms of "stupid".

Even in contexts where generic insults directed at people are not tolerated, it should be acceptable to recognize stupid ideas as such.