Show HN: OBS Live-streaming with 120ms latency
github.com
github.com
* low latency means you have a relationship with your audience. These intimate broadcasts are a new medium.
* Simulcast means it is way cheaper to run a streaming site. No more running ffmpeg/generating transcodes server side.
* AV1/H265/Opus means users with lower bandwidth can now broadcast. Users with enough bandwidth can stream at quality levels they couldn’t before
* UDP gives us IRL/Roaming streams. No custom setup for re-connects.
* Multi-track lets you send multiple video feeds or languages at once
* E2E Encryption means that P2P distribution could be a thing
Is that correct? I took a brief look a the WHIP protocol and it seems like maybe it's just a matter of converting the frames before writing?
* Lower latency - The extra decode + encode adds enough latency that you start to lose the real-time latency.
* Better Quality - You get generational loss from the transcoding. You get better quality only encoding one times. Streaming services also optimize for cost. Having broadcasters control the encoding quality of everything makes for a better experience.
* Security/Trust - Servers shouldn't be able to modify video at all. I would like for broadcast services to eventually offer E2E encryption. It feels wrong to me that a streaming service is able to modify video however it pleases.
This would significantly complicate services such as Twitch and YT Live going to server-side ad insertion if the source video were E2E. I think server-side ad insertion is likely the only method available for providers that isn't able to be circumvented by ad-block or DNS blocking plugins.
That said, you could use TLS from the server to client, which is what many services do (since it's actually quite difficult to use http these days for streamed content due to browser polices).
Is this even conceptually possible? I suppose you could sign the stream, but if you want to hide it how would you prevent the server from simply adding itself as a viewer?
(also, if you do this, start a countdown for getting raided as a CSAM distributor)
RTMP can't handle it, but SRT (and apparently WebRTC can). It would reduce latency for sure. Of course, it does require a good network connection on the client end, but so does streaming.
`--enable-features=WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled`
I haven't used it myself yet.
* Free Software
* Project is operated/managed by individuals
* Use Open Standards/Protocols
Sorry for hijacking your comment :)
I agree, I'm just struggling to differentiate what BB offers that MMTX does not, so I can identify if there's a USP. If it's a passion project to scratch a personal itch, that's also great!
Also, can you share your latency measuring methodology?
MediaMTX used to be "rtsp-simple-server" which really undersold what it was capable of (even back then it could ingest RTMP and output HLS and WebRTC).
Overall I think MediaMTX has more features and can do everything that Broadcast Box can.
You hit the nail on the head here. I mean live streaming is already quite big but it seems like it's going to be going up 10x - 100x next few years.
It's really an amazing thing that makes the Internet more human.
My goodness, you are living in the future. What else are you into? Where can I follow you (blog, twitter, warpcast, etc?)? Love to follow early adopters.
¹ https://developers.cloudflare.com/stream/webrtc-beta
² https://blog.cloudflare.com/webrtc-whip-whep-cloudflare-stre...
> Broadcast Box uses WebRTC for broadcast and playback. By using WebRTC instead of RTMP and HLS you get the fastest experience possible.
Nothing in RTMP prevents you from achieving low latency; it's the software stack around it that determines latency for both RTMP and WebRTC. Only HLS does have some built-in deficiencies that cause extra latency.
RTMP for me is preventing that next generation of use cases.
If you run into bugs or have any cool ideas join the discord and would love to chat https://discord.gg/An5jjhNUE3
Live streaming just seems to have so many downsides to me:
1. Requires real-time presence
2. No editing (meaning less efficient use of time for the viewer)
3. No client-side speeding up/skipping irrelevant parts
4. No possibility of index or table of contents
What are some use cases where live streaming is better?
There is also the "event" nature of streaming, where the audience is excited to react to things together in real-time, like when people Tweet when watching the Superbowl or a show's season finale. Even for mundane streams, the "chat" will happily talk amongst themselves reacting to whatever the streamer is doing, which can feel like hanging out with friends.
There's also times where the streamer is discussing something that is happening now or just happening (world events, the Superbowl, a game update, etc), where viewers are excited about the content right now and don't want to wait for the traditional record->edit->release cycle.
Streaming is also much more low-cost to produce, editing can often represent an unwanted source of complexity and loss of creative control.
Live streaming is inherently mor einteractive and there's a shared experience simply because you can't speed up.
I guess you could see that live-streaming is experience-based. VODs are results-based. Not strictly the case but there's a trend.
No one EVER interacts.
My 15-second YouTube Shorts of my cat get around 400 views, sometimes as many as 10,000!
Go figure.
There is a special skill of streamers who can play an action game on one half of the screen and read enough of chat on the other half to make and respond to jokes, at the same time.
Another consideration: youtube and twitch apply different music licensing considerations to live, so almost anything that involves music MUST be live and unarchived.
When I was more into the low latency streaming space a few years ago, it felt like WebRTC was there when it came to << 1 second latency, but the infrastructure to actually distribute it was not really. I think Cloudflare (and maybe some other vendors) were working on creating some standard, has it landed? Can I run my own horizontally scalable WebRTC broadcaster (are there open source implementations)?
Something like Low-Latency HLS or CMAF was at like < 5 second latency, but was on the other hand stupidly easy to distribute widely (just static files on a plain old CDN / http server).
Not knowledgeable here, but also interested if someone actually knows the answer to your question...
Quite a few companies offer this a service though. Cloudflare, Twitch’s IVS, Dolby Millicast, Phenix RTS…
LiveKit did a write up on how they built theirs https://blog.livekit.io/scaling-webrtc-with-distributed-mesh...
Why is that? Is that a protocol issue with WebRTC or an implementation issue with the way WebRTC servers are written?
There are lots of ways to share session state across servers.
`OBS -> Broadcast Box -> Broadcast Box -> Broadcast Box -> Viewer`
To start I would do WHIP between the servers as well. More optimization can be done, but it would be a good quick start.
i'm not going to get 120ms latency though. i'm in argentina, they're mostly in the usa, and i have 200+ milliseconds of latency over the internet to anything in the usa
if broadcast box isn't what i'm looking for, is there something else? i already know about zoom, google, and teams, but those all make us vulnerable to proprietary servers
I think you could run your own server, run an instance of the front end and distribute the instructions to setup OBS to your family.
Cloudflare if you want this quick. Even though it is proprietary you have no vendor lock in, just change your WHIP URL in OBS if things aren’t going right.
i suspect webrtc implementations in browsers are the source of many of the problems but i don't know how to debug that
Now I have to read the protocol. I was thinking the SFU was just a proxy, but it sounds more like it also handles some signaling.
This is from my T420 Thinkpad on Linux. The only significant input I can think is my RTT to the server.
``` Reply from 165.227.221.230: bytes=32 time=37ms TTL=49 Reply from 165.227.221.230: bytes=32 time=37ms TTL=49 ```
There've always been (a majority of) broadcasters that had seconds to half minute (!) delays. Very few understood, or even today understand, how to tune every single touchpoint (including what's safe to do, and what one must not do.)
Having an ultra low latency “video toaster” / broadcast mixer is a critical piece for sure.
For us the motivations to work on this started with "backstage feeds" synced with live TV that needed to look simultaneous to the home cable viewer (think World Wrestling Entertainment) as well as Wall Street calls that needed to be (perceptually) in sync with dial-ins. And of course, Times Square NYE ball drops!
In reality, almost nothing matters that way. For vast majority of content, the viewer is consuming only your stream, without some more real time channel at the same time. In most cases, the viewer cares far more about visual quality than the delay.
I hope if more broadcasters (technical and non-technical) realize what is possible they will ask for it more!
> For vast majority of content, the viewer is consuming only your stream
Agree! I think it is that way because you can't have intimate/interactive streams yet. When it becomes possible I hope to see more streaming to smaller/more connected audiences.
An HLS stream is so simple to set up with passable performance that few other protocols can compete.
We push technical advance until it hits a cliff in returns. We will never get there. There will be no 10 million mile drivetrain in a car. There will be no multi-generational fridge. There will be no house that withstands a tornado.
Software feels like it should be different because it's "free", but really any advance beyond the cliff is a great gift and not something to take for granted.
Even live news has large a large latency with remote interviews. Some of them are so bad it's uncomfortable watching the anchor have the patience to wait for a response before stepping on the remote feed.
Some streaming services are 30 seconds plus behind OTA, at that level twitter and half a dozen news apps have pushed the fact the goal went in while the ball is still in the other half.
It made the dream of ‘co-streaming’ available to home users and not just those with hardware/pro setups.
“You could also use P2P to pull other broadcasters into your stream. No special configuration or servers required anymore to get sub-second co-streams.”
I currently have this setup for doing a co-stream with a friend and it is terrible:
1. Friend is running OBS to capture his gameplay. 2. Friend has OBS streaming to a Raspberry Pi I have running at my house. 3. The Raspberry Pi is running nginx configured to accept the RTMP stream. 4. I run OBS on another machine to capture my gameplay, add overlays, etc. 5. My OBS has an input source using VLC to capture the stream from the Raspberry Pi.
The setup is awful. Video is pretty delayed and it often just stops working. I would love to look into this project but after reading through the README, I am unclear how I would use this for my setup. Any pointers?
Then I would pull that source into your OBS. You have two options
* Pull Broadcast Box as a browser source
* Use my PR that adds WHEP Sources https://github.com/obsproject/obs-studio/pull/10353
For now I would do Browser Source (is easier/no custom builds). In the future when WHEP is merged that is the way to go. If you get stuck jump in the Discord and happy to help debug! https://discord.gg/An5jjhNUE3
> I've never been able to use any WebRTC-enabled service for this reason, nor anyone else I know
Is this a browser/agent issue? You aren't able to use Google Meet or Zoom in the browser?
```
https://meet.google.com/***-***-\*\*, { iceServers: [], iceTransportPolicy: all, bundlePolicy: max-bundle, rtcpMuxPolicy: require, iceCandidatePoolSize: 0 },
```
But something like this is a selective forwarder, so as long as you can find someplace to host it that can get an ip:port to listen on, all your users/compatriots should be able to connect to it, and you're ready to go.
https://humanbenchmark.com/tests/reactiontime
Yes. Me clicking the mouse, the software transmitting the mouse click to a datacenter, software rendering the web browser using AVX2, software encoding the stream, sending it to the local browser and decoding it on the screen, shining photons in my eyes, me clicking the button a second time (which also needs to be transmitted to the datacenter), gets me around ~400ms over WebRTC on the reaction time benchmark vs ~200ms on the local computer. I'm not even trying. It's a janky as hell solution that is about to fall apart the moment you look at it funny.
Also, I hate ffmpeg for streams that last longer than a day. The latency creep of streams lasting weeks is horrible.
In the repo linked in OP is a screenshot showing a wall clock next to its playback on the streaming site -- that's end-to-end to me. So how is this relevant?
A proper implementation will make sure the worst-case latency is accounted for and not cherry-pick the best case.
While the page provides some insightful comments I don't see how it relates to implementation details short of 'they did not build a hardware card'.
Could you add more to your question? Where did you already do research? What providers are you considering? How much bandwidth is “free” with the tier of VPS you’re paying for? Etc
This is a question that sales reps spend lots of work hours trying to help clients work out.
I've used Moonlight + Nvidia Gamestream with ~40ms RTT and couldn't feel a difference in competitive shooters, so total latency must be pretty low.
Does it have something to do with the bandwidth requirements? (1 stream v/s potentially hundreds)
(For more details, I'm planning to stream from an old laptop after turning it into a hackintosh. I'm hoping staying on the home network's going to help with latency.)
I have all my PCs connected to each other via Moonlight+Sunshine and the latency on the local network is unnoticeable. I code on my Linux workstation from my laptop, play games on the gaming PC from my workstation, etcetera and it is all basically perfect.
With full hardware capture and encode (default on windows, can require tweaking on Linux ) it's virtually free resource-wise.
Bandwidth requirements are a big one. For broadcasts you want your assets to be cacheable in CDN and on device, and without custom edge + client code + custom media package, that means traditional urls which each contain a short (eg 2s) mp4 segment of the stream.
The container format used is typically mp4, and you cannot write the mp4 metadata without knowing the size of each frame, which you don't know until encoding finishes. Let's call this "segment packaging latency".
To avoid this, it's necessary to use (typically invent) a new protocol other than DASH/HLS + mp4. Also need cache logic on the CDN to handle this new format.
For smooth playback without interruptions, devices want to buffer as much as possible, especially for unreliable connections. Let's call this "playback buffer latency".
Playback buffer latency can be minimized by writing a custom playback client, it's just a lot of work.
Then there is the ABR part, where there is a manifest being fetches that contains a list of all available bitrates. This needs to be updated, devices need to fetch it and then fetch the next content. Let's call this "manifest rtt latency".
Lastly (?) there is the latency from video encoding itself. For the most efficient encoding / highest quality, B-frames should be used. But those are "lookahead" frames, and a typical 3 frame lookahead already adds ~50 ms at 60fps. Not to mention the milliseconds spent doing the encoding calculations themselves.
Big players are rewriting large parts of the stack to have lower latency, including inventing new protocols other than DASH/HLS for streaming, to avoid the manifest RTT latency hit.
IMO one of the issues is that transcoding to lower resolutions usually happens on the server side. That takes time. If the client transcoded that latency would go away (mostly).
OBS+WebRTC is mostly software doing heavy-lifting.
Imagine if the camera would build WebRTC UDP packets directly and zero-copy to NIC, that would lower latency quite a bit.
The tech is all there, it just requires some arcane knowledge.
Maybe if you’re playing Microsoft Office it’s ok.
You're not going to be able to do the best combos with that kind of latency, but I guess it's ok for mid-level play.