Comment by iseanstevens | Hacker News Reader

Comment by iseanstevens | Hacker News Reader

Latency should be around 110ms "glass to glass" based on my experience with "WifiBroadcast", which was an earlier (first?) thing to use Wifi in this way. (1) and a few related projects (2).

It's a pretty clever hack, using Wifi without associating, but still using FEC (Forward Error Correction) and mirroring that to allow bi-directional communications for control/telemetry. Also allows RF diversity reception with multiple wifi adapters.

I was experimenting with it for semi-local telepresence robotics for art/experiences. These projects are amazing in how much cheaper/more approachable/hackable they make all this vs. analog or proprietary digital video transmission.

I'm not sure if OpenHD is directly related to WifiBroadcast/ I haven't had a chance to play with it yet (been meaning to for 6 months+), but it's at least using similar approaches with wifi (mis)use.

There have also been some attempts to use the same approach with esp32 (etc) in a compatible way, though the link eludes me currently.

(1)https://befinitiv.wordpress.com/wifibroadcast-analog-like-tr...

(2) https://github.com/rodizio1/EZ-WifiBroadcast

Glass-to-glass latencies are extremely dependent on what cameras and displays are used.

Common 4G link latencies are 15-30 ms. With fast cameras and displays we can get sub-50-ms glass-to-glass over 4G.

For sure something like this should easily get lower than 110 ms, if care is taken to use fast cameras and displays.

WiFi links can easily achieve similar latency.

The cameras, displays, and interfaces used are certainly a factor in the overall glass to glass latency. There are also issues of codecs and hardware accelerated decoding/encoding of video.

Where “just use 4G” misses the point though is that this system is connectionless and the FEC that comes with that introduces some latency as well. This could probably be optimized somewhat as well.

The trade off with this type of system though is that you don’t 100% lose video for a considerable amount of time if the link is broken. When a two-way radio link is broken there is considerable latency waiting for the connection to be reestablished.

Interestingly, this technology can do 110ms glass to glass for video on generic hardware, but even "gaming" Bluetooth headphones (among those that don't require a special dongle) still cannot achieve 110ms of sound latency from software to speaker.

(Latency may be better or worse with OpenHD - I'm basing figures on EZ-WifiBroadcast)

Latency could be improve if we had access to low level parts of the RPI to modify camera encoding code, changing it from encoding whole frame at a time to for example encoding 4 separate picture strips. Same bandwidth, 4x lower encoder delay. Afair one of OnLive patents covered similar method. They patented every obvious thing, like for example dual encoding variable quality stream while recording high quality video - something every Twitch streamer does, but they also patented some actually useful stuff for lowering latency.

> Latency should be around 110ms "glass to glass"

A zoom or webRTC video call's latency ought to be 110 ms glass-to-glass.

The fact it isn't highlights design deficiencies at every level of our current OS stacks.

WebRTC video involves sending video segments not individual frames, which is the main contributor to the latency. The biggest problem with segments is overcoming saturation but this is a protocol issue. See https://snr.stanford.edu/salsify/

WebRTC is much, much faster. 21ms round trip “glass to glass and back” is the lower limit we demonstrated in practice from data center to headset with 3dstreamingtoolkit. There are a lot of reasons that you gain latency in large scale videoconferencing apps, but it ain’t because the tech is lacking.

Reply on news.ycombinator.com