Casting from my phone also got de-synced & broke a lot. I couldn't find a good way to skip/scrub forwards or backwards a little bit. Sometimes the themeing would break & the app became unusable without restart.
It was really really cool having jellyfin-mpv-shim running on my desktop, and Chromecast elsewhere. But Jellyfin was straight up not working for me, not listing a bunch of my media, not making it navigable, and the Android app's dodginess all ruined things for me. I went back to UPnP/DLNA, whose apps are a little cruder (Gerbera for MediaServer, BubbleUPnP for control, Rygel PlayBin for MediaRenderer), everything basically just works and it's baked into many devices/tv's.
Jellyfin fixed that. Other people in my house use it easily, a friend uses the web ui over Tailscale, and my messing-with-tech versus watching-things ratio is finally looking good.
The whole *arr thing makes it even better with automated download and upgrade.
But I can live without that.
The one thing it offers that folders do not is a simple way to interact with the video library when guests are using it.
They can grab a mouse and meander over the available shows with no explanation and watch what they want within minutes of being shown the link without exposing my nas to them or giving them the ability to alter my media files without my knowledge.