Hacker News Reader

Comment by VoidWhisperer | Hacker News Reader

> No head-of-line blocking: Unlike TCP where one lost packet blocks everything behind it, QUIC streams are independent. A lost packet on one stream (e.g., an audio track) doesn't block another (e.g., the main video track). This alone eliminates the stuttering that plagued RTMP.

It is likely that I am missing this due to not being super familiar with these technologies, but how does this prevent desync between audio and video if there are lost packets on, for the example, the audio track, but the video track isn't blocked and keeps on playing?

View on HN

Synchronized playback is usually primarily a player responsibility, not something you should (solely) rely on your transport to provide. We have had some talk about extensions to allow for synchronizing multiple tracks by group boundaries at each hop through a relay system, but it's not clear if that's really needed yet.

Essentially though, there are typically some small jitter buffers at the receiver and the player knows how draw from those buffers, syncing audio and video. Someone who works more on the player side could probably go into a lot more interesting detail about approaches to doing that, especially at low latencies. I know it can also get complicated with hardware details of how long it takes an audio sample vs. a video frame to actually be reproduced once the application sinks it into the playback queue.

If you’re delivering audio and video separately the blocking is irrelevant for needing to solve synchronization. That’s why some amount of buffering (a few frames of video at least) on the receiver is needed to hide the jitter between packets / make sure you have the video. You can go super low latency with no buffering but then you need to drop out video / audio when issues occur and those will be visible as glitches - depends on how good your network is.

Each track is further segmented into streams. So you can prioritize new > old, in addition to audio > video.

Depending on the streaming protocol (eg WARP), you can specify that the tracks (audio vs video) need to be time-aligned, so each group (chunk of video or audio) starts at the same time and lasts the same length. I think this means you'll get resync'd at the start of the next group.

Reply on news.ycombinator.com