Guilty as charged. That said: the noise itself isn't entirely just the problem so much as how prolonged it is. Having buses and readily-available public transport that are only loud occasionally and on a fixed schedule isn't the same as a constant stream of traffic. Even if the bus or train is louder (and light rail is usually much, much quieter), it isn't all-day, only at fixed intervals.
There are marginal gains to be had. Reducing the amount of driving and traffic in our cities is good for a lot of reasons, but particularly with noise it's a combination of speed, vehicle size, and overall frequency of vehicles that makes a killer combination.
The solution is to place noise restrictions on vehicles (this goes against manufacturers putting off any kind of noise reduction), reducing speeds (this involves walkable / bikeable infrastructure), and expanding public transit like bus and light rail access to reduce the total number of personal vehicles on the road.
Yes, this will make things "worse" for drivers. But it's not the end of the world. If you ever visit Zurich or Geneva, you'll be able to distinctly tell the difference between an American city and a European city in terms of noise alone :) Zurich's tram system is honestly something that every city should dream of having, and it is as quiet as one could hope for.