One problem is the fact that the codec needs to be negotiated, and if you're unlucky with codec compatibility, both callers fall back to crappy old codecs. Then there are tons of options for audio profile selection depending on requirements and bandwidth available (see https://en.wikipedia.org/wiki/Enhanced_Voice_Services for an overview) which makes it difficult to say what cause your specific problems.
Without VolTE, you're falling back to 3G audio, probably AMR or AMR-WB, which is quite old and doesn't compress as well as modern standards.
Unless you mean the headphone profile for Bluetooth headsets: that's terrible because the standard is ancient, back when Bluetooth had even less capacity for low latency data transfer, and the codec is suboptimal making the situation even worse. There are better codecs out there, and some headsets will support what some call mSBC, which massively improves the audio quality (but not exactly to a HD audio stream because of limitations). There have been several proprietary attempts to fix this issue, but implementing those solutions costs money so many headphones ship without them.
Cabled connections are superior to wireless ones, even more so because traditional landlines had dedicated connections and as such had no need to compress anything.