A lot of the problem is caused by cell phones. People talking on phones are usually louder than in-person chatter. And while a conversation is distracting, half of one is worse. Also, cell phone conversations on public transit seem to be more common than talking in-person. Everyone has a phone, but not everyone takes the train with a conversation partner.
Different cultures have different rules about mobile phones. While vacationing in Japan, I noticed that nobody talks on their phone on trains. It's wonderful. As soon as I came back to the states, I was crudely reminded how horrible American cell phone etiquette is.
A few years ago, I bought a cell phone jammer. It has paid for itself many times over. If you find yourself distracted by obnoxious people on cell phones, buy one. You won't regret it.