Comment by kmeisthax | Hacker News Reader

Comment by kmeisthax | Hacker News Reader

Parent Full thread

For those wondering how it works: compressed video streams can be arbitrarily interrupted in between frames by any compatible start point from another video frame. Decoding and reencoding is only necessary to change the contents of those frames or restructure the videostream to make it compatible.

Most video formats are a stream of delta-encoded frames that must be decoded and displayed in-order. If you want to go back even a single frame, you have to restart decoding from the beginning, and if one frame is damaged, you lose picture. So encoders have to regularly restart the video stream by inserting keyframes. And it's valid to insert keyframes basically anywhere into a foreign video stream, at least for the same codec and resolution.

Proper NLEs don't do this AFAIK - most of the cuts you do with them aren't approximatable with simple GOP[0] cutting tricks, and most people aren't editing video clips that are encoded to their final delivery format. Hell, at the professional level a lot of people record in ProRes, which is deliberately not delta-encoded[1], because delta encoding is actually really bad for nonlinear video editing.

[0] Group of Pictures - a keyframe plus all the delta-encoded frames up to the next keyframe.

[1] And has enormous size because of it

Maybe I can explain it differently?

---

Basically, lossy compression codecs store groups of frames, or groups of samples. You just can't "cut" at any frame or sample that you want. If you do, you have to re-encode the whole thing.

If you want to understand why, the "delta encoded" thing means that only the difference between frames (or samples) is stored. It usually takes less bits to store the difference between frames, or samples, than it does to store each frame / sample separately.

The reason why "groups" exist is so that you can jump around in a video / audio file. The beginning of each group stores a complete frame / sample. (This also helps if the file is corrupted.)

As long as you cut a compressed audio / video file at the groups / jumps, you can cut it losslessly.

---

(Note: Techniques like companding can use less bits when each frame / sample are stored separately. Some people don't even call this lossless. The technique is basically a digital version of the old "Dolby B" button on tape decks from the 1980s and 1990s.)

Some container formats, such as ISO Base Media File Format aka "mpeg4", have features such as edit lists. With edit lists you can have original video in the bitstream, organized in GoPs as you say, but then the final video rendered in the player gets to choose which frames to play from that material.

This allows cutting at completely arbitrary places (I'm 99% sure that this can also be at a higher time resolution than the original data), with the cost that the original (unplayed) data remains in the file.

However, some players may have incomplete support for them. For example I believe FFmpeg only supports edit lists used at the beginning of the video, which is commonly used to align video and audio properly.

> As long as you cut a compressed audio / video file at the groups / jumps, you can cut it losslessly.

If you had a video with keyframes 'd'K and delta frames 'd' like

  0           1  |<-->|   2
  KdddddddddddKdddddddddddKddddddddddd

and you wanted to cut out the indicated frames, could you create a new keyframe from the keyframe marked '1' + the two following delta frames, and then another new keyframe from K{1}dddddddd? Something like

  0           1  |<-->|   2
  KdddddddddddKdddddddddddKddddddddddd
              Kdd       = keyframe "L"
              Kdddddddd = keyframe "M"
  
  KdddddddddddKdLMddKddddddddddd

Or I guess if you're creating new frames that weren't in the original, could you just rewrite all the deltas in between K1 and K2 such that those 5 frames aren't there?

  0           1  |<-->|   2
  KdddddddddddKdddddddddddKddddddddddd
  KdddddddddddKddeeeKddddddddddd

(where 'e' marks recomputed deltas)

You're speculating quite a bit about what the codecs support.

See https://news.ycombinator.com/item?id=40844633. Some container formats support metadata so you can cut without reencoding.

How many frames per group, usually?

Varies by codec.

Reply on news.ycombinator.com