Why don't you plug your nice headphones/headset into the computer and pair your phone bluetooth to the computer? That way you can listen to audio from your phone and your computer without swapping jacks.
It's too bad, especially since the SRS-X3 was not a cheap purchase. And NFC speaker pairing is all sorts of cool. But music that stutters constantly? The crappiest $2.50 earbuds you can find make for better listening than that.
Less problems with CyanogenMod 12 and 13's stack, though.