Apple’s Low Latency HLS differs from the community-developed solution

WebRTC options are improving, check out

https://github.com/ant-media/Ant-Media-Server

You want WebRTC

I wonder why serving an endless http stream of something like h264 is not an option? Can't ffmpeg produce it in real time at your resolution?

You should probably try mixer. They rolled a low latency protocol by their own. It use websocket as a bilateral channel to allow the server push whatever it want to client directly. Achieving sub second delay (The model here looks more like webrtc instead of hls though)

Check out NDI. Near real-time local video streaming. It’s inexpensive/no cost if you are doing it all in software.

Does it need to work in a browser or could you provide VLC or similar as the client and use the 90s protocol that actually addresses your use case?

Similar situation here, ended up with the same solution, after an initial attempt with HLS. jsmpeg (https://github.com/phoboslab/jsmpeg) made it pretty easy.

> I looked a Flash: but of course it's not desirable to use it these days.

RTMP protocol has a lot of implementations and is still widely used for the backend part of transmitting video at a low latency (i.e. from the recorder to the server).

RTSP with or without interleaved stream is another option.

DASH/HLS is a solution for worldwide CDN delivery and browser based rendering. Poorly suited for low latency.

If you need low latency and browser based rendering you need something custom.

How does Twitch do low latency mode? Isn't that 1-2 seconds?

I have no idea what the underlying tech is, but Steam Link can do extremely low latency on the same network and very low latency over the internet. It can also stream non-game applications, though I imagine automating steam is a nightmare.

You can also consider tunneling over WebSockets. It's a lot easier than WebRTC especially you don't need the handshaking nonsense which often require self hosting STUN and TURN servers if you don't want to rely on third parties. IIRC the performance of WebSockets is good enough for companies like Xoom.

https://webrtchacks.com/zoom-avoids-using-webrtc/

Some VNC services like noVNC and Xpra also uses WebSockets.

A fairly long time ago (3-4) years I was tasked to do something fairly similar (though running on Android as the end client). HLS was one of the better options but came at the same costs you describe here. However it was fairly easy to reduce the block size to be less to favor response vs resilience. Essentially you trade buffer size and bitrate switching quality for more precise scrolling through the video and faster start times.

I had to hack it quite severely to get fast load with fair resilience for my usecase as the devices are restricted in performance and can have fairly low bandwidth. Since you're looking at a relatively fast connection, simply reducing the chunk size should get you to the target.

As a follow up - I've spent a couple years working on a video product based on WebRTC. This either works for a PoC where you just hack things together or on a large scale where you have time and resources to fight odd bugs and work through a spectrum of logistical hoops in setting it up. So unless you plan to have a large-ish deployment with people taking care of it I would stick to HLS or other simpler protocols.

Why can't you simply use plain HTTP streaming directly loaded by the browser with video tag?

You should try webrtc. googles stadia is built upon webrtc, so I assume it should be able to give latency in miliseconds.

Me and my friends have our own little streaming website and manage to get 1~2 seconds delay.... It's nothing fancy, NGINX with the RTMP plugin from which we get the streams, it only passes them trough, once we added encoding we had a noticeable delay. This is flash tech that can be run as html 5 now, but I didn't see this within your list so perhaps you haven't looked at it.

Try streaming TS packets over Websockets and decoding with FFMPEG compiled to WASM in the browser. I wrote, https://github.com/colek42/streamingDemo a couple years back, and despite the hacky code it worked really well. You could probably do much better today.

We recently completed a project with similar requirements. We ended up using rtsp from the camera and packing it up in websockets using ffmpeg. We had sub second latency. The camera gave h264 so we could just repack that. We're giving a talk about the project on MonteVIDEO Tech meetup, though it will be in Spanish.

Contact me if you want to discuss it further.

There are good reasons CDNs don't support http/2 push. It’s hard to load balance and hard to operate, since it requires a persistent TCP connection with the client for the entire duration of the stream, which can be hours. It has consequences that echo almost everywhere in the architecture.

What exactly is the benefit of HTTP2 for HLS CDN use, particularly?

The obvious benefit of not using it is that you don't need your CDN to do TLS, likely to be utterly superfluous if video chunks are validated through the secure side-channel of the playlist already.

The main redeeming feature of traditional HLS is that it can use ordinary HTTP CDN infrastructure. If you're going to require video-streaming-specific functionality in CDNs anyway there is absolutely no justification for designing your protocol in this horrendously convoluted, inefficient, poorly-performing way.

We also lost the ability to broadcast things like rain, snow, fireworks, dark scenes and confetti with the loss of analog.

Some delay from many producers is almost certainly intentional. Live content providers want to be able to have a second to cut a stream if something unexpected (profanity, nudity, injury...) occurs on set.

Google seems to think they can implement video gaming over IP. And they probably can, my ping to them is only 9ms, less than a frame.

There is just a broad lack of interest in reducing latency past a certain point unless there is a business reason for it. People don't notice 1 second of latency.

Yes, and its driven by consumers.

Unless your content operates in a very small niche, "real time" is far less important that continuity.

In rough order of preference for the consumer:

1) it starts fast 2) it never stops playing 3) It looks colourful 4) Good quality sound 5) Good quality picture 10) latency

one of the main reason why "live" broadcast over digital TV has a stock latency of >1 second is FEC. (forward error correction) this allows a continuous stream of high quality over a noisy transport mechanism. (yes, there is the local operating rules for indecent behaviour, and switch and effects delays, which account for 10 seconds and >250ms respectively)

For IPtv its buffer. Having a stuttering stream will cause your consumers to switch off/go elsewhere. One of the reasons why realplayer held on for so long was that it was the only system that could dynamically switch bitrates seamless, and reliably.

There is a reason why netflix et al start off with a low quality stream, and then switchout to HD 30 seconds in, its that people want to watch it now, with no interruption. They have millions of data points to back that up.

> Now the best we can do is over 1 second

Mixer can do about .2 seconds.

It's not surprising if you think about how our ability to store video has changed over the years. The delay on analog TV is so low because the picture data had to go straight from the camera to the screen with basically no buffering since it was infeasible to store that much data. (PAL televisions buffered the previous scanline in an analog delay line for colour decoding purposes, but that was pretty much cutting edge at the time.) Now that we can buffer multiple frames cheaply, that makes it feasible to compress video and transmit it without the kind of dedicated, high-bandwidth, low-latency links required in the analog days. Which in turn makes it possible to choose from more than a handful of channels.

Analog TV is also massively less spectrum efficient. You can fit 4+ digital channels in the same spectrum as one analog TV channel.

And don't forget how low and inconsistent the quality of analog TV was compared to what we can broadcast digitally.

The real story here is that latency isn't actually important to live TV, so it's a no-brainer trade-off to make. If you look at other transmission technologies where latency is more important, like cellular data transmission, latency has only decreased over the years.

Thanks, that's exactly how I felt — that there’s a really good and useful article in here, but clouded by assumptions and an attempt to create controversy.

Note: HN altered the title from the original to which my comment refers.

To your last point,

> Maybe the current "implementation" is compatible but not with the specs itself.

It's perhaps worth noting that this is a "preliminary specification" and an extension of HLS. HLS itself is an IETF standard (well - an "Internet Draft"): https://tools.ietf.org/html/draft-pantos-http-live-streaming...

> from the time the last bit of the playlist is returned to the last bit of the video segment

Based on my loose understanding of HTTP/2 server push and ALHLS, the sequence of events will be:

1. Client requests playlist for future media segment/"Part"

2. Server blocks (does not send response) until the segment is available

3. Server sends the playlist ("manifest") as the response body along with a push promise for the segment itself

The push then begins with the segment.

The push stream can presumably occur concurrently with the response body stream. So I don't think you can wait until every bit of the playlist comes in. Likewise, you can't use the playlist bits itself to gauge bandwidth because the server imposes latency by blocking.

> As usual, Apple pushes NIH, instead of supporting DASH which is the common standard.

I mean... HLS predates DASH. It would've been hard for them to support a common standard which didn't even exist at the time. Initial release of HLS was in 2009[0], work started on DASH in 2010[1].

I'd also disagree with the characterization of DASH as "the commmon standard" - it's certainly a legitimate standard, but I feel like support for HLS is more ubiquitous than support for DASH (please correct me if I'm wrong).

[0] https://en.wikipedia.org/wiki/HTTP_Live_Streaming

[1] https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_ove...