Electric cars that don't make much noise seem more dangerous to me as a pedestrian.
At higher speeds you can here something moving - there are tyres rolling at high speed over a stone-like surface and that's inherently noisy no matter if there's an engine of any sort.
EVs won't affect the noise problem, they'll just cause the assholes to change their tactics.