The sound from a phone speaker is annoying, more so, than a typical in person talking. To me the solution lies somewhere in fixing that to make it sound more natural.
Everyone else claiming that some how having “loud” conversation is rude, feels like they’ve fallen into some anti-social hole… we are literally the only animal to have developed complex spoken language… it’s part of our humanity.