Hacker News Reader

Comment by EGreg | Hacker News Reader

Traveling in Europe currently, but got curious.

How many viewers can it support? RTMP usually goes to platforms that broadcast to many users. What about with WebRTC, can I have 100 viewers peer to peer? Maybe with TURN relays?

View on HN

Practically: however many your computer and uplink can serve. OBS needs to be tested, but this browser to browser test may give some indication: https://tensorworks.com.au/blog/webrtc-stream-limits-investi....

If you've got 100 viewers and can manage to saturate a gigabit uplink (which can be difficult because of routing and other upstream crap) you should be able to send about 9mbps video + overhead, which is pretty close to Twitch's bandwidth cap.

Because consumer ISPs generally don't have great routes between them and sustained gigabit uploads probably don't always work reliably because of overbooking with consumer lines, you'll probably be better off setting up an SFU on a cheap (temporary) cloud server somewhere.

Theoretically: media streams are identified by a 32 bit number, so about 4 billion clients.

Data streams are limited to 65535 endpoints (a signed 16 bit number with a reserved 0).

Not sure why you think that? Maybe on cable modems, because DOCSIS is highly asymmetrical, but on FTTH GPON et al (pretty much the only technology which supports gigabit upload; docsis gigabit upload is extremely rare), there is no reason it couldn't saturate upstream at 1gigabit+. If anything, consumer ISPs are usually way more contended on the downlink side than the upstream side.

> Maybe on cable modems

Maybe I'm wrong but I'm pretty sure cable modems are still wha the vast majority of consumer households have.

Yes but the parent is referring to gigabit upstream connections. Very few gigabit upstream services are over DOCSIS. I don't know of a single provider that offers gigabit upstream on docsis. In the UK Virgin Media offer 100mbit/sec up on their top tier (1gigabit down). I think the fastest Comcast goes in the US is 200mbit/sec up over docsis and that's only in certain areas.

GPON is typically shared bandwidth; 2.5Gbps for n customers (that could be an entire building, or the 4 houses adjacent to a box in the backyard, etc.)

More advanced fiber networks split bandwidth based on frequency ("color") instead of giving each ONT a time slot based on the current network capacity. But basically, if you're uploading at 1Gbps, that is fine until 1.5 more neighbors decide to do the same thing. It's rare. When I worked at Google Fiber we kept an eye on PON utilization and there was never a case where a customer couldn't get the requested bandwidth. We even had customers that maxed out 1Gbps/1Gbps 24/7 in an attempt to get banned or something, but of course we wouldn't ban you for that. We did worry about PON capacity and had some tech ready to go in case we ever needed to switch to truly dedicated per-customer gigabit connections.

At another ISP I worked at, our marketing material described the 1Gbps GPON-based link as "dedicated", but many customers noticed it was shared. It would slow down during peak business hours, and then get fast when other people in the building went home from work. We removed the term "dedicated" from our marketing. A political battle that I paid dearly for, but hey, they still exist and didn't get sued into oblivion by the FTC, so that's nice. (We also sold frequency-multiplexed 10Gbps circuits that really were dedicated... I suppose until our uplink to our ISP was saturated. I think we had 100Gbps transit and that never happened. But yeah, there is no such thing as dedicated unless you are physically plugged into the network that you want to reach. The Internet tries its best.)

I know that GPON is shared; but as you say it's rarely the 'weak link'. Regardless, even if it was, it's much more likely that the downstream would be saturated on the gpon level vs the upstream - consumer ISPs are much more downstream heavy than upstream heavy, so I have no idea what the parent post means when he says that its hard to saturated gigabit uplink on consumer connections. It's the opposite if anything.

It’s not about bandwidth inside the AS, but about fiber peering from ISP datacenters “to the Internet”. I don’t have anyone’s internal informations but that must cost 10-100x more than $35/Gbps/month. Residential internet is gym membership model, a best effort offering. Resources are always heavily over-committed based on typical use cases of groups of average users.

“consumer ISPs generally don't have great routes between them” is the way more important part.

An eyeball network will generally have considerably more, and considerably higher bandwidth, peering relationships with content provider networks than with other eyeball networks.

At AVStack we experimented quite a bit with P2P multiparty video calls, it’s certainly possible but as you scale up the number of participants there’s huge variance in quality depending on the combination of ISPs used by the participants, the time of day, network conditions etc.

What do you mean by Twitch's bandwidth cap? Obviously Twitch supports more than 100 viewers, so is this an upstream cap from the streamer or a downstream per user cap or something different all together.

A glace over Twitch's broadcasting guidelines was not enlightening, but I'm clueless in these matters.

He means the cap on the bitrate of the video that twitch will ingest. Twitch will then send this video to any number of viewers, as you say.

If you send more than about 6.75mbps of video to Twitch, you'll start getting errors. With the variable size of encoded streams you need to play with the cap a little to get a problem-free experience.

That's an apples to oranges comparison. With proper infrastructure both can scale to many users. For streaming to many people peer to peer will be inefficient, so I would recommend not doing that.

How can OBS with the new WebRTC and WHIP+WHEP be used to stream to thousands of people at once, without using RTMP?

And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?

How can a webserver serve 10k people at once?

Cloudflare Stream has beta support for webrtc ingestion and playback. If you did more research you may find other services or projects for scaling webrtc streaming to handle many viewers and handling transcoding to lower quality for people without enough bandwidth.

>And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?

It looks like a pool of ingestion servers and a CDN for viewers. Ingestion servers handle getting the stream from the streamer and encoding it into whatever formats are needed. The CDN handles distributing this stream to people who want to watch it.

I have talked with companies that are already doing 100k+. I don't have the exact numbers but Millicast and Phenix both do very large audiences.

How are they doing it is the question? What's the set up?

In practice this is what SFUs are for. You would simulcast a set of different encodings from OBS to the SFU and that would fan out to other SFUs and clients as appropriate until the desired scale is reached. This does get expensive.

If you don’t actually need the low latency then you are better off with hls-ll delivered via a CDN.

If I'm ingesting and decrypting the stream anyway, wouldn't it be better to build an MCU, to send out one stream and adapt the resolution for each client?

https://www.digitalsamba.com/blog/p2p-sfu-and-mcu-webrtc-arc...

Cost. SFUs don’t have to transcode the video, just pass buffers around and ensure both ends agree on the format. That is an enormously lighter process.

Personally I think simulcast (sending multiple differently encoded streams) is almost always wrong for many Webrtc uses (especially mobile publishers) but in the OBS case it actually makes a lot more sense.

How does WebRTC know what resolution to send? Or does it always send the original resolution?

That all depends on how you set it up. Generally speaking a lot of WebRTC is about how the connection negotiates the parameters of the delivery of the different media, including codecs, resolution, etc. One track can be delivered over a connection in multiple different ways at once as well.

If you want to know more you're probably best off going through the MDN docs about the WebRTC browser API and learning from that.

Reply on news.ycombinator.com