> I know video cards have sepcialized hardware for video encoding, but those are tuned for realtime streaming and not meximum quality. If you want to use a different algorithm than the one baked into the hardware, can you use a GPU for that?

If you're doing something that follows the same general design as most existing video codecs, absolutely; these days "hardware" encoding is less "fixed function chip" and more "the driver/firmware knows how to stitch together these components so as to comply with the xyz standard" - hence you see support for new encoding standards getting added in driver updates. If you're doing some completely radical out-of-left-field video compression algorithm, maybe not, although even then GPU hardware is pretty general these days and video encoding tends to be well suited to running on it.