A single uncompressed frame of 1080p video occupies 28MB in RAM, so 1 minute of 24fps video will take up 40GB. If you want to be able to run 4 cores at once it's 3 times that. You won't be doing that any time soon on your laptop or smartphone.
So I still do not see how this would prohibit parallel processing.
you cut the video into a handful of parts at keyframes, process the parts individually in a streaming manner and then splice the partial results together.
If we're talking about playback then creating seeking-thumbnails could similarly benefit from parallel processing.