How much math are you comfortable with? Codecs are about packing the most information into the smallest energy, which is a bit of a task.
I'm currently refreshing on math topics because of that, I do not mind very technical resources.
I find there is not a lot of information online regarding low-level details of video codecs, having any resources is greatly appreciated.