I have this issue on a new MBP 16, and it seems like something forces the Bluetooth codec to switch whenever the mic on the BT device is active.
Bluetooth has many modes. The regular audio one is A2DP which sounds good but has no mic support. Then there is HSP which sounds like a tin can microphone broadcast over AM radio. I imagine apple has something proprietary to do 2 way audio on the airpods tho.