Plex just uses ffmpeg, the version bundled does include GPU support although I believe it depends on the codec, from memory it does x264 but not x265 or vp9 but I could be wrong.