I have Jellyfin setup alongside Plex, and the one thing that's keeping me from moving fully off Plex is Plexamp [1]. It works flawlessly on my phone/laptops, includes a ton of features and is a pleasure to use - especially compared to Spotify or Apple Music.
I am aware of FinAmp [2] and music players for Jellyfin, but they aren't comparable to PlexAmp yet.