OpenHD - HD video, UAV telemetry, audio, and RC control | Hacker News Reader

OpenHD - HD video, UAV telemetry, audio, and RC control

github.com

216 pointsdanboarder34 comments

Latency should be around 110ms "glass to glass" based on my experience with "WifiBroadcast", which was an earlier (first?) thing to use Wifi in this way. (1) and a few related projects (2).

It's a pretty clever hack, using Wifi without associating, but still using FEC (Forward Error Correction) and mirroring that to allow bi-directional communications for control/telemetry. Also allows RF diversity reception with multiple wifi adapters.

I was experimenting with it for semi-local telepresence robotics for art/experiences. These projects are amazing in how much cheaper/more approachable/hackable they make all this vs. analog or proprietary digital video transmission.

I'm not sure if OpenHD is directly related to WifiBroadcast/ I haven't had a chance to play with it yet (been meaning to for 6 months+), but it's at least using similar approaches with wifi (mis)use.

There have also been some attempts to use the same approach with esp32 (etc) in a compatible way, though the link eludes me currently.

(1)https://befinitiv.wordpress.com/wifibroadcast-analog-like-tr...

(2) https://github.com/rodizio1/EZ-WifiBroadcast

(Latency may be better or worse with OpenHD - I'm basing figures on EZ-WifiBroadcast)

> Latency should be around 110ms "glass to glass"

A zoom or webRTC video call's latency ought to be 110 ms glass-to-glass.

The fact it isn't highlights design deficiencies at every level of our current OS stacks.

WebRTC video involves sending video segments not individual frames, which is the main contributor to the latency. The biggest problem with segments is overcoming saturation but this is a protocol issue. See https://snr.stanford.edu/salsify/

WebRTC is much, much faster. 21ms round trip “glass to glass and back” is the lower limit we demonstrated in practice from data center to headset with 3dstreamingtoolkit. There are a lot of reasons that you gain latency in large scale videoconferencing apps, but it ain’t because the tech is lacking.

Glass-to-glass latencies are extremely dependent on what cameras and displays are used.

Common 4G link latencies are 15-30 ms. With fast cameras and displays we can get sub-50-ms glass-to-glass over 4G.

For sure something like this should easily get lower than 110 ms, if care is taken to use fast cameras and displays.

WiFi links can easily achieve similar latency.

The cameras, displays, and interfaces used are certainly a factor in the overall glass to glass latency. There are also issues of codecs and hardware accelerated decoding/encoding of video.

Where “just use 4G” misses the point though is that this system is connectionless and the FEC that comes with that introduces some latency as well. This could probably be optimized somewhat as well.

The trade off with this type of system though is that you don’t 100% lose video for a considerable amount of time if the link is broken. When a two-way radio link is broken there is considerable latency waiting for the connection to be reestablished.

Interestingly, this technology can do 110ms glass to glass for video on generic hardware, but even "gaming" Bluetooth headphones (among those that don't require a special dongle) still cannot achieve 110ms of sound latency from software to speaker.

Latency could be improve if we had access to low level parts of the RPI to modify camera encoding code, changing it from encoding whole frame at a time to for example encoding 4 separate picture strips. Same bandwidth, 4x lower encoder delay. Afair one of OnLive patents covered similar method. They patented every obvious thing, like for example dual encoding variable quality stream while recording high quality video - something every Twitch streamer does, but they also patented some actually useful stuff for lowering latency.

It's just so unfortunate that there's so much focus on p2p/distributed internet protocols, and so little focus on local/nearby connectivity.

This is a very very rare example of someone not assuming the now-classic role of connectivity being an access point plus clients, and it's for such good, such a basic advancement.

Both AirDrop and Nearby Share have captured the market on another form of local connectivity, but both rely upon deep deep integration with centralized identity service, both are locked to a very limited set of OS'es. Yet both are built around semi-standard specifications that few others have tapped or tried to use. That used to be wifi-p2p or the proprietary-itized Wifi Alliance wifi-direct spec, but now there's also Neighbor Aware Networking (wifi-nan), which is yet another amazing capability basically untapped except by a couple remote few.

We just need more torchbearers. wifi-p2p has been semi usable by anyone, but never gotten far. neighbor aware networking alas has been a proprietary thing, only accessible to the paid up expensive Wifi Alliance members, what a waste. but trying to re-assess, trying to advance connectivity, beyond the narrow spectrums, beyond the deliberate, small bounds we have wifi connectivity now: it seems like such an obvious moral imperative. and, as shown here, there are some sincere technical advantages to exploring the alternatives. not relaying all traffic through a central access point can be a good boon. yet that's where we've been morally stuck. i think we need to stop relying on products & software, need to do like these chiefs, & advance ourselves.

beyond the latency advantage, i would love so much to have some means to hail & open ports to those nearby & about me.

I was reading some Arduino docs for Bluetooth LE and it might do some of what you want.

https://www.arduino.cc/en/Reference/ArduinoBLE

https://github.com/catid/loraftp

LoRa is interesting & I love it's capabilities, but it's also one of the most proprietary least accessible forms of wireless communication we could pick, and the whole market is powered by basically one company's chips. Lora is in many ways to me a nightmare scenario versus the values & exploration I am talking about, even though it enables new modalities.

I’m very impressed by the video quality and the bitrate. The DJI FPV systems sending HD video use either 25 or 50 mbit/s, while this is getting comparable quality with an order of magnitude less throughout. It has me wondering if the 25mbit/s is a large amount of error correction bits sent in order to improve latency.

Two questions I have - what is the video latency for this system versus a DJI FPV system? And how many devices can be supported concurrently from a single access point?

It would be exciting if this tech could evolve to support “crowded fields” for FPV races.

Regarding fpv racing and low latency - OpenHD is more of a two-way network connection + HD video so the latency is around 135ms with a lot of optimization and OpenGL overlay graphics, while the shark byte HD FPV system is more optimized for very low latency (5 to 30ms) digital broadcast (I think sharkbyte is open hardware but not software): https://www.fatshark.com/product/shark-byte-module/

Open.HD latency testing thread: https://www.rcgroups.com/forums/showthread.php?3154184-Open-...

The problem is the video codec not being degradation tolerant, this way you need a quite an overkill amount of FEC.

If a codec can tolerate just a few percents of arbitrary bit errors without completely garbling the image, you can slash the bandwidth many times even with a youtube level HD quality.

The issue is that video compression (to produce good compression) creates anti-redundancy - it puts more information into fewer bits by adding dependencies for more areas onto a few bits, and merging many duplicate (redundant) bits into one spot. It also makes later frames more dependent on earlier frames and the side effects of decompressing, to avoid transferring the information again (and adding more anti-redundancy).

With a real-time feed used for command and control of something like a fast moving aircraft in close proximity to objects, you have the opposite situation - you want maximum reliability, clear and predictable degradation, and fastest recovery, even if it comes at the cost of clarity or bandwidth.

They’re literally working to opposite goals.

If you would like to see OpenHD in action, here is a 21km flight on an INAV setup: https://youtu.be/8uhfhsWnhBE

And here is a 4.5mbit vs 2.5mbit comparison: https://youtu.be/WrMAHqCEGqY

Hmm. That looks hardly any better then a very good analog feed with zero latency.

I beg to differ, it looks quite a bit better than analog, even past the fact that "good" analog at that distance is already hard to achieve.

https://youtu.be/npxUsrbcdoY

>Instead, Open.HD configures the Wi-Fi adapter in a way that is closer to a simple broadcast, much like analog video transmission hardware you may be using already.

Would this be able to broadcast a video which multiple clients could playback in sync? In the 90s, when you were watching TV, if you had a TV in the kitchen and a TV in the living room, if both were tuned to the same channel, they'd play back exactly in sync. Would this project enable something similar?

i think the issue there is that modern displays decode the signal at different rates, depending on their changing hardware (HDMI/display technologies). with old TVs, the signal was not really decoded, but projected directly onto the "glass". so i imagine it would still not be in sync unless you have two of the same TV.

Modern TVs (anything that doesn’t have an analogue-only path all the way to the display, which is probably almost every TV sold in the last 10-15 years) are designed in a way that (unintentionally) disregards the importance of predictable signal-to-visibility delays.

Basically this is because all modern displays have a digital path from their (primarily digital) inputs which always (even if it’s “disabled”) includes signal processing to convert the incoming signal (of varying resolutions, refresh rates, color spaces, etc.) to signal(s) which can be sent to the LCD (either through the TCON embedded in the side of the LCD or through some custom ASIC that directly generates the analogue waveforms to drive the matrix of Liquid Crystals), backlight driver(s) (because artificially-measured contrast ratio numbers can be seriously enhanced for marketing), audio outputs, etc.

That’s on top of the extra image processing features that many TV’s advertise (“smooth motion”, “200hz”, etc.) which generally require the video DSP to buffer one or more frames to perform the processing (and maybe generate intermediate frames). These additional features can often be disabled by enabling a “gaming mode” which is (usually) designed to reduce perceptible input latency but this is not always implemented correctly by the manufacture and most reviewers don’t perform end-to-end latency measurements so there’s not much incentive for manufacturers to do much better than “ok”.

Analogue-only CRT TVs didn’t have nearly the same complexity (or features) in their signal processing and could reliably be trusted to display a given input with a minimal delay, mostly due to the fact that they didn’t have any (significant) form of memory to buffer signals for processing.

If I recall correctly, there was an era in analogue broadcast where the signal that was generated in the studio broadcast camera became the synchronisation signal for every TV that was tuned to that channel which essentially means that a TV tuned into that broadcast would actually scan the electron beam across the CRT at exactly the same rate and time as all the other TVs on the same channel. This required careful signal design to ensure that a “cheap” TV with poor timekeeping could still synchronise its operation with the incoming signal but it was far more practical/affordable than trying to buffer an entire frame given the technology at the time.

To answer your question directly, it might allow you to have synchronised broadcast to multiple TV’s but it may require playing with the output resolution/refresh rates of your OpenHD receiver and with the settings on the TV (try looking for a “gaming” picture preset).

This is pretty neat, I have just finished wiring a basic RPi zero / camera / GPS / barometer setup [1] for an electric RC glider and have been wondering about real-time video. Cheap FPV systems use essentially analog TV NTSC broadcast which limits the quality quite a bit. I have been wondering if I could use off-the-shelf wifi hardware and use it with some kind of broadcast protocol instead and they do exactly that! Eager to try it out if I manage to get the right adapter.

I also considered using a USB LTE adapter and just push the video with RTMP which would make the entire setup much less complex (although with some data charges) but coverage is a bit of a problem in rural areas where I normally fly.

1] https://heap.ovh/raspberry-pi-zero-glider-loadout.html

If anyone interested to interact with OpenHd Devs you can join the ExpressLRS discord and talk in the dedicated OpenHD room at this address https://discord.gg/dS6ReFY

Edit SOmeone reported it's not a dedicated openhd room. But anyway, lotsa peoples working on DIY HD FPV system there. Come join the brainstorming and the fun :)

https://t.me/OpenHD_HDFPV

Looks like they're using RPi. Curious to see if there's a smaller board for done racing.

Typical latency doesn't seem to be listed. Although the connection latency is probably low enough, I'm curious about the video encoder / decoder latency.

The DJI system is the state of the art consumer unit basically. With the caddx unit it's especially small (to fit on smaller quads):

https://www.quadcopters.co.uk/dji-hd-digital-fpv/iflight-suc...

The silver box and the camera are the relevant parts, the connection to the other PCB is just to receive telemetry from the flight controller.

Edit: better link to the actual unit itself:

https://www.buildyourowndrone.co.uk/caddx-fpv-air-unit-micro...

Can I pave my dgi phantom 3 with this and make it fly?

I think DJI is pretty locked down. Probably you would want to build a new drone, OpenHD uses a Raspberry Pi and ExpressLRS uses STM32 (and a few other various chips) but you can get DIY kits that are relatively easy to set up on sites like https://www.getfpv.com for example.

Also check out INAV if you want to build a more DJI-style drone, this would be the Flight Control software: https://github.com/iNavFlight/inav

Other FC software include ArduPilot for more of a groundstation approach, while Betaflight, EmuFlight, and similar FC software are what FPV race, freestyle, and cinematic pilots prefer.

This is correct, but to clarify something, you don't need ExpressLRS for OpenHD, they're entirely independent (one is for control, the other for video).

Also, Ardupilot blows INAV completely out of the water for reliability, at least for fixed wing craft. I've switched all my planes to it from INAV.

What constitutes a failure in reliability for this type of software? The drone falling out of the sky? Flying off in some random direction for no visible reason?

In particular, INAV has problems with keeping the position of the horizon accurately, and usually thinks it's going straight while it's dramatically losing altitude. So, "all of the above", really.

The biggest issue is that this usually happens during automatic modes, which are things like "return to home", which you usually use when you've lost control or video and you really want RTH to work reliably. I can't risk a plane falling onto a car or person, so I use ArduPilot, whose auto modes are impressively accurate.

Reply on news.ycombinator.com