The flicker would be a challenge but you can use something like DeOldify [1] or super resolution [2] on the frame sequence to improve the colour and resolution... although you may get some weird artifacts between frames using image models rather than models built for video.

Generally video models are quite far behind image models because they're so much higher dimensional (much more computationally expensive and expensive to annotate) but I'm sure it's a matter of time before someone releases a DeOldify type model for video (if one doesn't exist already).

[1] https://github.com/jantic/DeOldify [2] https://github.com/idealo/image-super-resolution