It's actually worse than that - first you have to tune, then you have to wait for a PAT frame (which has a index of PMTs in it), then you have to wait for a PMT (which contains pointers to the audio, video and ECM streams, and then you have to wait for an ECM (encrypted key for the stream), at that point you have have decrypted video and can start looking for I-frames ....

(smart systems both cache a whole bunch of this stuff and revalidate their caches on the fly while they are tuning - first tunes after boot might be slower as these caches are filled)