What sort of hardware are you using to do transcoding of the content on the source box? You could try adjusting the quality (or type) of the transcoding to potentially improve this. Like, perhaps you are trying to transcode into a format for which your video card doesn't have built-in encoding? (Like, nvenc units on an nvidia chip, if that's what you have.)

You could also try setting up directplay, if your stack supports that.

Lastly, it might just be a limitation of your host hardware. What are you running your jellyfin server on?