Aspartame.
Yet people still hate it.
It tastes bad to the average male, it leaves a bad after taste that is really noticable. Females don't seem to notice the after taste.
Now, for phenylketonurics it's problematic, but they have severe dietary restrictions anyway.
The only problem I'm aware of with aspartame is that consumption of any sweetener is associated with weight gain, possibly due to changes in the brain's association between taste and calorie intake.