The bluetooth asics tend to be very single function oriented. They might support multiple protocols on the stack but they are generally missing things like a muxer so only one audio based protocol at a time. That means if you're streaming audio using A2DP and you receive a voice call, the A2DP stream is dropped and HSP or Handsfree is enabled. The host will then transition to streaming over the new lower bitrate connection.
The asics generally don't support concurrent connections from different devices, with a few exceptions, and so there's a lot of scaffolding that has to be broken down and stood up when switching devices. It's not unlike HTTP over TCP. There probably needs to be a SPDY for Bluetooth.
Some semiconductor manufacturer could really steal the market by making a bluetooth asic with an audio muxer that supports multiple concurrent protocols and can effectively blend the audio while also maintaining multiple concurrent host connections.
From a UX perspective, this presents problems with situations like listening to music on a headset connected to a PC when an audio call comes from your phone. Does the headset duck A2DP? Send PAUSE over AVCTP if available? Just MUX the audio streams and let you sort it out? What do device controls control when connected to two devices?
Another issue with Bluetooth is that hosts generally don't surface useful controls for prioritizing devices or protocols. So in the case of being paired with multiple A2DP devices, a host will usually only automatically connect to the last one used and you have to explicitly connect to others. This is annoying if you regularly transition between Car and Headphones.
There's also an issue of multiple hosts connecting to the same client and prioritization. Hosts tend to open and hold onto a connect even when they're not doing anything because of the scaffolding involved in a connection. That means if you have two phones and one car, only one will work and if you want to use the other one then it requires terminating the other host's connection, usually on that host device.