All JPEG decoders have chroma upsampling. Here jpeg2png only seems to actually optimize it though; seems like it works but optimizing for "smoothness" could maybe be improved.

I'm suggesting something like eedi3 (http://avisynth.nl/index.php/Eedi3) but guided by the luma plane.

Oh, you can also jointly optimize the YUV-RGB conversion because if the pixel ends up out of range in RGB space it is probably a compression artifact. I read a paper on this once but haven't seen it implemented.