OBS merges WebRTC support
github.com
github.com
* Serverless Streaming - WebRTC is P2P so you can video right into your browser. You don’t have to stand up a server anymore to stream for a small audience.
* Sub-Second Latency - Create content and interact with viewers instantly. There is something magical about having a real conversation with your viewers.
* Multitrack Input - Upload your transcodes instead of generating then server side. Give viewers multiple video tracks to see action from all sides.
* Mobility - WebRTC lets you switch networks at any time. Go from WiFi -> Mobile with zero interruptions.
Try it out today with Broadcast Box. A reference server implementation. https://github.com/glimesh/broadcast-box. The PR to add WebRTC to OBS https://github.com/obsproject/obs-studio/pull/7926
-----
So many people went into making this happen. This was a colossal undertaking that different people have working on for over 6 months.
Sergio Murillo - Created WHIP. The only reason this could be added to OBS.
Luke Strickland - Created Broadcast Box. The reference server we developed against
Paul-Louis Ageneau - Creator of libdatachannel. The library used to add WebRTC support to OBS.
Colin Edwards - Started the project to add WebRTC into OBS
John Bradley, tt2468, pkv - Developers who worked on the code itself
RytoEx, tytan652 - Lots of feedback and reviews
-----
Have fun using it! If you have any questions/feedback/improvement ideas I would love to hear.
How many viewers can it support? RTMP usually goes to platforms that broadcast to many users. What about with WebRTC, can I have 100 viewers peer to peer? Maybe with TURN relays?
And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?
>And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?
It looks like a pool of ingestion servers and a CDN for viewers. Ingestion servers handle getting the stream from the streamer and encoding it into whatever formats are needed. The CDN handles distributing this stream to people who want to watch it.
If you don’t actually need the low latency then you are better off with hls-ll delivered via a CDN.
https://www.digitalsamba.com/blog/p2p-sfu-and-mcu-webrtc-arc...
Personally I think simulcast (sending multiple differently encoded streams) is almost always wrong for many Webrtc uses (especially mobile publishers) but in the OBS case it actually makes a lot more sense.
If you want to know more you're probably best off going through the MDN docs about the WebRTC browser API and learning from that.
If you've got 100 viewers and can manage to saturate a gigabit uplink (which can be difficult because of routing and other upstream crap) you should be able to send about 9mbps video + overhead, which is pretty close to Twitch's bandwidth cap.
Because consumer ISPs generally don't have great routes between them and sustained gigabit uploads probably don't always work reliably because of overbooking with consumer lines, you'll probably be better off setting up an SFU on a cheap (temporary) cloud server somewhere.
Theoretically: media streams are identified by a 32 bit number, so about 4 billion clients.
Data streams are limited to 65535 endpoints (a signed 16 bit number with a reserved 0).
Not sure why you think that? Maybe on cable modems, because DOCSIS is highly asymmetrical, but on FTTH GPON et al (pretty much the only technology which supports gigabit upload; docsis gigabit upload is extremely rare), there is no reason it couldn't saturate upstream at 1gigabit+. If anything, consumer ISPs are usually way more contended on the downlink side than the upstream side.
Maybe I'm wrong but I'm pretty sure cable modems are still wha the vast majority of consumer households have.
More advanced fiber networks split bandwidth based on frequency ("color") instead of giving each ONT a time slot based on the current network capacity. But basically, if you're uploading at 1Gbps, that is fine until 1.5 more neighbors decide to do the same thing. It's rare. When I worked at Google Fiber we kept an eye on PON utilization and there was never a case where a customer couldn't get the requested bandwidth. We even had customers that maxed out 1Gbps/1Gbps 24/7 in an attempt to get banned or something, but of course we wouldn't ban you for that. We did worry about PON capacity and had some tech ready to go in case we ever needed to switch to truly dedicated per-customer gigabit connections.
At another ISP I worked at, our marketing material described the 1Gbps GPON-based link as "dedicated", but many customers noticed it was shared. It would slow down during peak business hours, and then get fast when other people in the building went home from work. We removed the term "dedicated" from our marketing. A political battle that I paid dearly for, but hey, they still exist and didn't get sued into oblivion by the FTC, so that's nice. (We also sold frequency-multiplexed 10Gbps circuits that really were dedicated... I suppose until our uplink to our ISP was saturated. I think we had 100Gbps transit and that never happened. But yeah, there is no such thing as dedicated unless you are physically plugged into the network that you want to reach. The Internet tries its best.)
An eyeball network will generally have considerably more, and considerably higher bandwidth, peering relationships with content provider networks than with other eyeball networks.
At AVStack we experimented quite a bit with P2P multiparty video calls, it’s certainly possible but as you scale up the number of participants there’s huge variance in quality depending on the combination of ISPs used by the participants, the time of day, network conditions etc.
A glace over Twitch's broadcasting guidelines was not enlightening, but I'm clueless in these matters.
How is this possible?
also this is probably only taking California as the whole world, like everyone does.
If you are curious on the 'how' of WebRTC I wrote a Free/Open Source book that goes into the details https://webrtcforthecurious.com/. Happy to answer any particular questions you have.
I added WebRTC support (audio, video, chat) to my already existing application a couple of years ago and felt the technology wasn't exactly ready for primetime. To me, it felt like the out of the box developer experience wasn't exactly great.
I am only saying this because I got to see exactly how much effort is required to get things going. Your work is greatly appreciated
The numbers are purely illustrative. I don’t know what OPS can actually do but let’s say each OBS server can handle 50 connections that means if we had a two level deep tree, we could handle 2500 participants in a single call.
For incoming connections and compositing them you need something beyond OBS. Gstreamer is one option.
For my own usage I run WebRTC servers and have them 'replicate' the video traffic. LiveKit did a write up on this with some good graphics https://blog.livekit.io/scaling-webrtc-with-distributed-mesh...
Anyway it’s super exciting stuff! Great work!
It has a docker image so you should be able to run a 'streaming site' with just a few commands :)
As I understand webrtc, it describes the connection wrapping and the streams / negotiation, but not the discovery part. Which means it's about as p2p as http - if you add a way to find the endpoint and open sockets, you can make peers talk.
Is there something more in the OBS implementation that makes it more p2p? Am I missing something?
it's not really meant to be a useful tool for making a pure peer-to-peer video service.
I have a chapter on signaling here https://webrtcforthecurious.com/docs/02-signaling/
I have an example of WebRTC without signaling https://github.com/pion/offline-browser-communication. Each side has to agree on things ahead of time. Useful for security cameras/IoT/LAN stuff though!
`localhost` is allowed. Could you reverse port forward the service to localhost? That will get around those restrictions.
Meanwhile I can hack together a poor man's webrtc by sending raw packets over websockets and playback works just fine. I just lose all the real benefits of webrtc (robustness, udp vs tcp, etc.). It feels like it should be possible to work around "secure context" nonsense since I don't access anything on the clients, they just play back a stream from the devices. But chrome/ff tends to block the ice handshake in http mode.
But yeah, one way or another, you need a secure context for WebRTC to work. That’s in the spec, and browsers follow it.
Perhaps in your context it really is nonsense, but how is the browser to know that your network/physical layer is secure?
That discovery mechanism can be a decentralized network where any contact is an entrypoint. You connect to the network ~once and from that point onward everything is p2p and no fixed servers are involved or in control.
Usually you'd divide what you're talking about into two different steps. Peer discovery, and peer transport.
Discovery is how you find other peers, sometimes its via DHT, sometimes it's hard-coded, other times just random walk. But just like in WebRTC, (and in TCP), you need to get the address from somewhere out of band, so you can connect, usually something like a centralized directory. But the transport of data, is P2P, just like TCP. So once you've found the peer and connected to it, all data goes directly between you.
Doesn't make it less/more P2P than any other P2P system out there.
https://b.siobud.com might not be always available. So best to host yourself for anything important :)
Excited for you to use it. Send any ideas my way for improvement.
I've been trying it out for a while now (streaming to a self-hosted broadcast-box), and almost everyone who views the stream says that the video will freeze for 1 to 2 seconds every now and again (every few minutes). Audio is uninterrupted. Any hints about how I can debug this?
Here's some speculation: Most users will experience non-libwebrtc WebRTC bugs (like OBS users). 1% of users who experience bugs in OBS WebRTC will report enough telemetry to fix the bug. But 99% of all of OBS's userbase uses Chrome, Edge or Safari, and maybe 50% of those (more than enough) report telemetry. So if their WebRTC bug was hardware / network specific - the bugs that matter - libwebrtc will have observed it and by now, probably fixed it.
Despite telemetry from hundreds of millions of users every day, it still took years for libwebrtc to reach truly robust, flawless stability on all the platforms it deploys on.
It's a complex technology.
chrome://webrtc-internals should tell you the exact reason. My guess is packet loss and then it recovers on the keyframe.
In the next day or two I am gonna open a PR to set NACK buffer properly. Right now it is too small :/
Sorry you are hitting issues. Will try my best to fix it up! Email me sean@pion.ly and would love to make sure we get it working 100%. I bet other users are hitting the same issues and not sharing.
I've always wanted this. Instead of the streamer switching video inputs we could select from the viewer-side which perspective we want. I've also thought about things like NASCAR partnering with Valve/Steam to use their Valve Index for 360-degree views from each car on the track. I don't know why they're not marking VR to people who love NASCAR, it'd be such an odd and likely successful niche. It'd be cool to accept-in multiple video inputs and even patch them together in realtime on the viewer side (unless they're specifically disparate).
* Serverless Streaming - WebRTC is P2P so you can video right into your browser. You don’t have to stand up a server anymore to stream for a small audience.
> Only possible in an ideal world, where NAT doesn't exist, in reality, you need to traverse NAT and it is NOT a serverless process.
* Sub-Second Latency - Create content and interact with viewers instantly. There is something magical about having a real conversation with your viewers.
> Only possible in an ideal world, where machines are very close to each other. In reality, you need expensive world-wide TURN clusters on top-notch infrastructure to ensure <100ms latencies you want.
* Mobility - WebRTC lets you switch networks at any time. Go from WiFi -> Mobile with zero interruptions.
> In fact, an interruption is happening when network conditions change – since you need to negotiate connectivity again. It is not seamless.
[1] https://www.cisco.com/c/en/us/support/docs/ip/network-addres...
I have a DIY WebRTC forwarder for a RTSP stream, that lives on a home server behind a NAT. It has NAT because it lives in a cgroup that's isolated to IoT VLAN, and I haven't originally planned on WebRTC there, hoping I could make it work with restreaming over HTTP or Websockets. The NAT there is of the most common type: nftables' masquerade statement for the appropriate output interface, the usual conntrack allow established,related rule, and rule that allows outbound connections. For whatever reason, WebRTC only worked for me when my viewing device wasn't behind any other NATs.
Now, this is not a proper argument. Being a very lazy ass I haven't bothered with diagnostics so I don't know why it hadn't worked. I was already quite frustrated with various cross-browser JS shenanigans and after checking that STUN was indeed configured but had not worked, I've just punched a couple of port forwards and called it a day.
For whatever reason P2P streaming seem to work somewhat worse in practice than it should've been in theory. Usual computer leprechauns, I guess.
——
Yes <100ms is hard, I don’t think most users are looking for that. 400-600ms is what people can expect and I always see that!
—-
You don’t need to negotiate again with NICER
400-600 ms is certainly not a "real conversation" experience, yet getting close to it. Speaking more realistically, the latencies spread would be much more broad depending on the peers' geography in a true P2P mesh. So with a part of the audience closer to streamer, the conversation can become more real indeed, but for those further away, it will be increasingly more choppy.
If you don't need to negotiate again, how do you know which new socket to send bytes to? Is it some kind of a wizard, this NICER?
Now this is actually HUGE. It means Twitch has far less power as a gateway for greenfield streamers.
Things you're giving up as a greenfield streamer by rolling their own streaming platform:
* The beginning of a follower base on your destination platform (Twitch)
* A "single pane of glass" for viewing your stream, chatting with you and other viewers, and giving you subscription/donation money. This is a deal breaker for a non-negligible number of people, and your follower base might not grow as large or as quickly as you'd like.
* Twitch Prime subscription money.
WebRTC support being added to FFmpeg
https://news.ycombinator.com/item?id=36130191
10 days ago
... how I can be a contributor of such p2p stream without a big tech web engine? (I use only noscript/basic (x)html browsers).
I could setup a "node" (should be plain and simple C coded) on my desktop computer, then from this node, I would extract the stream itself and watch it locally (I have a 800MBits uplink, and I know what IP diffserv with its ethernet translation is).
If the specs/protocol stack are reasonable and not too much convoluted, I would even be able to code such node (based on ffmpeg, very probably).
AND... if this p2p network is actually good and works IRL, do you really think the CDNs and streaming platforms will let it exist without a little bit of DDOS-ing...?
I will be adding input also. Making it easier for people to build streams like Twitch's Guest Star[0]. Just implementing the WHIP/WHEP, then we are going to see if we can add some batteries/better experience that is OBS specific.
[0] https://help.twitch.tv/s/article/guest-star?language=en_US
Still excellent still a huge win, but also a pretty big distion. Users still need considerably more to actually get online.
Same conflation happened for ffmpeg getting WHIP. https://news.ycombinator.com/item?id=36130191
WHIP is WebRTC signaling only. It just standardizes using a HTTP POST to exchange Offer/Answer. No media flows over HTTP.
[1] https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html
Conferencing providers just need to add WHIP support. It is trivial to add.
That would be awesome for consumers... but horrible for the business model of conferencing providers which is why it won't happen.
I think that will be unavoidable unless HTTP3/QUIC or IPv6 gains widespread support on all kinds of network infrastructure.
The best compromise would have been a dual stack: the clean pure p2p ipv6/tcp, and the brain f*cked ipv4/nat/upnp/turn/stun/etc. To have a clean and brutal split between the 2.
In my country, people would mostly run the clean ipv6/tcp protocol stack. And I am thinking REALLY simple: no transcode in the protocol (only the node will decide if it needs to transcode), one stream, buffered (it's not a video call). Yeah... excrutiatingly simple, then with many alternative implementations, aka sane.
But, it is kind of a dream, I know CDNs and streaming services won't let that happen in any way, I would not be surprised to see some DDOS-ing "from" them to keep people slaves of their big tech "centralized" services.
Back in 2001 your computer was probably connected directly to the internet over Ethernet with a dedicated IP address so the simple setup worked fine.
But then Wi-Fi came out and people had routers at home and NAT became the norm. This forced STUN/TURN.
It's entirely a networking issue, not a web issue. IPv6 could have made NAT a thing of the past, but obviously it hasn't. But firewalls are also the norm now in a way they weren't back in 2001 because security has to be tighter now.
The issue is the browser unable to deal with gateway UPNP... well since the web engines are now real cluster f*ks of everything, they could add a DOM interface for it.
Browser permissions to open up a port through UPNP is the "easy" part. The hard part is how your router will distinguish that as a legitimate request, versus local botnet malware opening up a request. There are some extensions around UPNP to add user authentication as part of the process, but they don't seem to be widely adopted. To the contrary, general security recommendations are usually to disable UPNP.
I had thought that videoconferencing would indeed finally be the impetus to solve this in a secure way, but meetings with much more than 2 participants require a centralized server anyways to prevent bandwidth from exploding, so relaying video calls through a server just became standard.
It has been like that for years. We are talking tens of millions of people with native IPv6, mobile and domestic.
NAT is something which should not have existed in the first place.
how "convenient" this "security risk" is in favor of big tech centralized services... yes, how convenient...
Guess what: I will gladely take "this security risk" on top of the already tons we already have, that, to open the gate to indenpendence from big tech centralized services.
And as I said, in my country, it has been years with tens of millions of people on ipv6, mobile and domestic.
"Security" is just police in the digital world. Police is a permanent process. Without digital police, we'll get digital anarchy.
You're free to not use a firewall but you'll certainly understand why most people aren't going to follow your example.
You cannot ask domestic users to deal with a firewall. It's beyond them, this is quite unreasonable actually. Most people don't even know what is a firewall.
We are talking about a digital freedom space which has to be protected by digital police or we'll end up with digital arnachy and in the filthy hands of the digital mafia which is big tech (seems to be already the case though). The other way around is digital dictatorship, which is no better.
And as I said, it is much better to go p2p and federated than to be jailed in big tech centralized services.
But I don't see big tech to let that happen, they'll probably hire some hackers in order to sabotage it. firewalls would actually serve them well: anything beyond centralized services would be "blocked". As I said, convenient, very convenient... way too much actually.
In my country tens of millions of people have IPv6, not to mention that gateway UPNP for IPv4 is almost everywhere too, then ten of millions of people have been "following my example" for years. Of course, without an efficient digital police, this will turn to an omega sh*t show.
People who work for centralized services which could be threaten by p2p protocols are more likely to try to scare away people from them.
No we're not, nobody said that.
> NAT is something which should not have existed in the first place.
It was an absolute necessity when it was introduced, since there are less IPv4 addresses than there are people on earth.
Maaybe there's some unnecessary incidental complexity in the way the STUN and TURN protocols work? I don't know, I honestly haven't investigated the details of the protocols. But the actually problematic complexity comes from the fact that something like STUN and TURN is needed at all, and that's essential complexity which arises from the problem domain.
It sounds to me, that you don't know what problem is being solved, so therefore you don't understand why the solution looks like it looks like.
:)
Instead, I use another app called ShareX [1] which is much lighter on the OS and processor. It may not have all the features but you can easily create a screencast or recording session with ease.
OBS is not a screen capture utility.
I've been looking for a way to record my screen and webcam as separate files and so far OBS¹ seems like it might be the only tool that can do that. Is that not a good use for OBS?
¹ https://obsproject.com/forum/resources/source-record.1285/)
In fact the issue was closed without understanding what the reporter was asking: https://github.com/obsproject/obs-studio/issues/8822
I log in into X11 when I use OBS on Ubuntu.
If you've got that set up up correctly, screen sharing will also work in Firefox (for instance on discord).
As far as I understand it, xdg-desktop-portal is a DE/WM agnostic protocol that enables applications to easily capture a screen - the user just has to run the right backend for their environment. I think it does other stuff too, but screen recording is probably the main use case.
I'm using Manjaro Sway Edition where that was configured out of the box.
ShareX - screen capture and screen recording.
OBS - streaming (and lots of related functionality).
I'm guessing you were using OBS for screen recording - but that's not really close to it's core feature set.
OBS' ability to compose a whole scene, dozens of different capture methods, overlays, full audio mixer, transitions, etc. It's the perfect place to record any sort of desktop-based video. It's far far more than just "screen recording".
I use it for streaming as well, but having it installed, I also use it for recording when I need it. Also for video capture from the webcam.
OBS can do no wrong to me.
ShareX doesn't live stream as far as I know? Let alone offer WebRTC?