Keyframes, InterFrame and Video Compression (2016)
blog.video.ibm.com
blog.video.ibm.com
In fact, with very very tiny modifications (ie. modifying the bit of the spec that says 'frame 1 must be a keyframe' and replacing it with 'the first frame is initialized with solid black'), I believe any modern video compression scheme could be used with no key frames. Compression ratio wouldn't suffer, but you'd loose the ability to seek the video.
Imagine we have a 1mbps link between us. We'd like have fluent video conferencing so we really want to have under 200ms latency. If keyframes are 10x the size of non-keyframes we'll be stuck either massively short changing the bitrate on keyframes, underutilizing our link (say using a 100kbps video), or taking a large amount of delay so that the keyframe's usage could be averaged out over the next $key_int frames.
Instead, if part of the frame is constantly being updated you'll still recover from loss in finite time, but the bitrate will be more constant.
> Compression ratio wouldn't suffer, but you'd loose the ability to seek the video.
Just having none means you can't recover from loss-- and in any case where you care about latency, you'll likely have some loss because retransmission creates delay (and forward error correction consumes bandwidth).
The solution to that is "non displayed frames", which VP8 allows. It means you can have a key frame which isn't part of the displayed video feed. You then send a super low res keyframe, and keep increasing the quality with further non-displayed frames until the quality is good enough to use as a reference frame for the real displayed frames.
That effectively lets you spread the bandwidth of a key frame over many seconds without introducing latency. The downside is both the encoder and the decoder need to do more computation to decode all those extra frames.
Modern video codecs tend not to be self-stabilising. For example, most codecs use some kind of huffman tables or arithmetic coding. If the probabilities for those tables depend on previous frames, and you don't know the probabilities, the resulting decode will be entirely junk, and won't get you any closer to being able to decode the next frame either.
When this kind of rolling keyframe is used the encoder needs to track the reference structure of the video and not make any references into the "dirty zone"-- this includes constraining coding modes so that no entropy context leaks. This can be aided by (or necessitate the use of) 'tiled'/'sliced' entropy coding features that some codecs have for parallelism reasons.
There is, of course, a coding performance loss-- especially if the video pans against the update boundary.
In x264 land it's apparently called "periodic intra refresh":
https://web.archive.org/web/20101017210542/http://git.videol...
The frames are divided into slices where one strip of the frame contains complete blocks, but the rest is motion predicted.