Have you tried PipeWire? As long as bt driver for you adapter is decent PipeWire makes using all sorts of bluetooth audio devices a breeze. At least for me it works great, all the time.