Aspartame has been extensively studied and no adverse effect has been found. Why do you think that? Is it because "scary chemical"? And is it worse than sugar?
Now, for phenylketonurics it's problematic, but they have severe dietary restrictions anyway.
The only problem I'm aware of with aspartame is that consumption of any sweetener is associated with weight gain, possibly due to changes in the brain's association between taste and calorie intake.