Try this just for fun: Call someone right next to you via your cell phones and talk to each other. One finds one can speak several words, maybe even a short but complete sentence before the other person starts hearing any of it. It can become a laughably long latency when experienced so directly.
But this cumulative latency ( be it from queuing delays, voice packet grouping for burst transmission, buffering delays along the many packet switches from here to there, you name it ) comprise engineering trade-offs necessary for optimal radio-level multiple access, efficient packetizing, and economically efficient (packet) transmission.
In other words, I'd postulate we can't (today) afford full duplex, sidetone-included, wideband (5KHz+) sub-20-millisecond latency in cellular telephony. Cellular is a different thing, but surely it's its own kind of magic.
But consider modern office telephony over Ethernet: I don't know the internals of it, but aesthetically, I find it significantly better than the analog telephony I remember from 40 years ago: The sidetone's there, it's full-duplex, speakerphone functionality is absolutely superb, and subjectively I find it sounds good.
I'd guesstimate a three orders of magnitude latency ratio between LAN-based telephony vs cellular telephony seems to me key to the very different engineering possibilities and compromises necessary in these two very different domains.