Hacker News Reader

> Twitch is almost a monopoly

Calling it Twitch, rather than Amazon, makes it sound better. But really, it's Amazon that's the monopoly - and they're able to use their bandwidth pricing power to prevent anyone else (aside from other mega-scale cloud providers) from getting anywhere close.

This is a great point that I wish was made more often.

Why would you be hosting this on AWS? Host it on a normal cloud provider that doesn't rip off it's customers with bandwidth fees and you'll find it's quite affordable.

Cloudfront isn't a great option here, but CDN transit fees are still pretty high for this sort of scenario unless you have significant economies of scale.

As some other people have pointed out in the thread, P2P delivery is a great use-case for live video distribution. I'm not convinced the discovery latency and overhead from p2p is worth it to distribute simple HTML pages, but for video it's just perfect.

I've heard of single instances of peertube running on a small VPS scaling to hundreds of livestream viewers. The ability to scale "instantly" by leaving a web browser to seed is a plus. I'm not sure how cross-instance redundancy/seeding works with livestreams though.

P2P is notionally interesting and I can see some edge cases for it but consistency of edge delivery are considerable and out of the publisher’s control, while the latency problems (which, you-don’t-need-that dismissals aside, are actually what people building stuff in this space care about) seem nearly irreparable by design.

A browser isn’t an option using current technologies; WebRTC doesn’t really scale like this. RTMP/SRT/WebRTC for ingest, HLS or equivalent for consumption. That’s what it’s all built around and expecting other people to donate transit to you isn’t a reliable model for folks who have other options, I think. And most people do have other options.

While I agree that it's not cheap, no idea why one would choose Cloudfront for live video which is pretty down there in the choices of a CDN for live video.

With Bunny you'd be at what, not even a $100.

> unlikely to be economical in practice

> your have 300 viewer on average

> a healthy 30 hours per week

I get that live-streaming on twitch is a whole thing, but many streamers would be really happy to get 30, or even 3 viewers, and don't stream anywhere near 30 hours a week. If you have a good home connection with a decent, no-cap upload, there's no need to pay a cloud provider either.

> If you have a good home connection with a decent, no-cap upload, there's no need to pay a cloud provider either.

serving viewers directly from your home IP means they know your home IP, and there's definitely already trends both to fuck with streamers generally, and specifically in games to DDOS people and get them dropped from the game.

Usually the mechanism for leaking IP data is voice chat (since that's often implemented via P2P because game companies are cheap) but you're talking about making your home IP available to everyone who connects to your stream.

"Your computer is broadcasting an IP!" advertisements aside... there are scenarios where that's a problem, specific IPs can certainly be DDOSed off the internet and that means a streamer won't be able to stream anymore.

> since that's often implemented via P2P because game companies are cheap

You implement it peer-to-peer for lower latency, not because you’re cheap.

I think the benefits that twitch provide go beyond just bandwidth. There might be enough bandwidth on a good home connection but onboarding users, building a community, getting sales to advertisers to support the service, etc are all core function that would be quite difficult for a sole provider to handle.

Other than preventing some viewers seeing a delayed feed, why do I need a CDN at all? This can run off a single server with a 1Gbps uplink... can't it?

In any case there's plenty of people/organisations who run less regular streams and don't need to serve this many viewers.

Latency and scaling, yeah. Depending on stream setting and encoding you might saturate hardware before bandwidth.

therefore renting a dedicated server from a provider with 100TB traffic included is still useful - there are many of them [1] around ~70€/month

[1] https://www.google.com/search?q=100+tb+trafic+server

Something like Hetzner will also work for 30 euro/month and certainly won't run out of bandwidth. Plus this isn't a great amount of traffic, getting a decent VPS should work fine.

I think the times of €30/month are over, Hetzner is now a €40/month provider (unless you take some hardware from the auction - I did and it was mostly fine, but with much lower specs than their regular offering).

How about peer-to-peer delivery like with PeerTube? It introduces a 30s-1m delay, which may not be acceptable for some streamers, but in exchange the users watching also participate in distribution.

A lot of the fun from live streaming is directly interacting with the streamer in chat. 30s-1m variable (I presume this is HLS torrenting?) latency would dash all the fun.

On the other hand, it may be a feature for some streamers:

"Our service includes built-in stream-sniper protection by default!"

I think a cheap streaming solution using bittorrent would be awesome. You would still need a beefy seed host but otherwise I'd love to something like that exist.

For a lot of cases, 60s of latency is fine.

For streaming I think a big part of the experience is 'chat' all seeing what each other are commenting on the thing that is happening right now, and the streamer's interaction with chat. Throw in a 30-60s delay that is variable depending on user and you destroy that element.

It just depends. Obviously "Twitch plays X" sort of content where there is a lot of audience participation isn't great with that much delay, but there are a lot of streamers where chat is more reactive than suggestive and a delay there just doesn't matter much.

You may want to add in an artificial delay to chat though so it's synchronized to the video.

This does exist, by the way: https://joinpeertube.org/

Your math is off by an order of magnitude. That pricing is for web apps. For streaming, look towards Bunny CDN and similar for wholesale pricing.

What you're describing isn't very realistic, since nobody would willingly choose the most expensive option(s) for transfer -- and CDNs aren't even particularly useful for live video, as the minimal latency increase of using a single location is easily counteracted with larger buffers, all of which is already baked in to the protocols.

Here's some counterpoint math including revenue, using even higher bitrates across a couple cheap load-balanced 4Gbps VPSs. For simplicity, I'll use a single average stream.

Transfer: 300 avg. viewers * 6 hours * 5mbps = 4TB @ $0.01/GB = $40

Revenue: 5,000 total views * 0.01 conversion * ($5 avg. contribution - processing) = $228

That's a $188 margin self-hosted versus approximately $120 via Twitch mechanisms. Presuming 5 streams per week, Twitch is taking a $17,000/year commission. Twitch does provide some other functionality and very minimal discovery, but if you already have a dedicated audience, then self-hosting absolutely makes sense for any sized streamer, especially considering that you're not limited to streaming to only one platform.

> you just stream a healthy 30 hours per week

Unless streaming is somehow your job, I fail to see how that is even remotely healthy.

"somehow"... Twitch pays about $3.50 per 1,000 views to full-time streamers, starting somewhere around 500 regular viewers. Top end streamers make $5k/mo streaming 40 hrs a week. It's not SF senior dev money, but high-school me would think it's pretty cool to be making money "just" playing video games. (Instead I'm posting on HN for free.)

Top-end streamers make > $1m/year.

https://dotesports.com/streaming/news/full-list-of-all-twitc...

For many popular streamers it is indeed their job

My last company wanted to self-host their own conferencing platform, bandwidth was one of the many reasons I told them it would be a horrible idea. Economies of scale are capable of solving many "simple" problems, and gain many benefits, due to their sheer size.

That could be true, especially if one live streams 24/7. It might be a nifty proof of concept if they set up a live demo without a CDN on a bog-standard VPS provider like Linode, DigitalOcean, Vultr, etc... Does anyone here have this running in one of the VPS providers?

Here's an instance running on Linode: https://live.retrostrange.com/

List of instances: https://directory.owncast.online/

Nice! What is the highest number of viewers you have ever seen on a stream at once?

I just found it in the list, wonder the same.

Phil here, I run the RetroStrange TV channel. We’ve seen hundreds but not sustained over time. Average currently 2-15 viewers at a time. I’d love a stress test.

Running the numbers for 300 streamers at ~3Mbps each for ~30 hours a week works out to 12,150,000MB. That value * 4 for about monthly bandwidth usage. For a s-4vcpu-8gb instance on DO, that's ~$436 monthly overage on a $40 monthly cost droplet. Not too insane, but you're really stressing out that single droplet. You'd probably want to balance that across a few different instances across the country instead of it all being on a single droplet, in which case you're essentially rolling your own probably crappier CDN and trading CDN costs for your time managing it all.

Agreed that was my thought process as well. One could spin up a few dozen instances, start the live stream and then nuke the instances. That should scale to however many viewers one has. I guess that begs the question does this application have the instrumentation or hooks that would make it super simple to scale up nodes on demand? Like an API to query number of viewers and their bandwidth settings to calculate required capacity.

For ~300 streamers, wouldn’t a single symmetrical fiber 1gbps line, like a residential AT&T install, basically cover it?

The TOS prohibits it but I always figured it’d be fine so long as one didn’t get too popular. I guess I’m wondering to what extent increased residential bandwith keeps CDN bandwidth cost in check.

Streamers won't want to serve directly from their home IP because it makes them a DDOS target.

Ddos prevention never crossed my mind, good point. I don’t spend much time in the gaming community. Who would be the most likely attacker, a competitor in the game trying to get an advantage? If so, thats kinda fascinating in a way. It’s almost like a cyberwar scenario where the battlefield is virtual and physical.

it's both, competitors in the game (or just trolls) have attacked random in-game players using IPs leaked from things like VOIP chat in games (which is often implemented P2P rather than running through the server) and people also specifically go after streamers to harass them.

https://www.eurogamer.net/dead-by-daylight-streamers-are-bei...

Other than the DoS attempts the sibling mentioned, the symmetric Gigabit on a home connection varies depending on congestion, and NAT mapping could change changing your IP causing the whole Livestream session to renegotiate.

I've livestreamed with ffmpeg and Icecast fanned out to a layer of cheap VPS and that worked ok. Yes there was enough lag to be annoying but not anything like 40 seconds.

Also interested to know this. How many people watching can a cheap VPS like Linode handle?

It'd likely involve a huge amount of latency, but any chance you can get a livestream working with torrent.js in the way PeerTube does? Maybe using HLS or something? Would certainly address some of the bandwidth concerns.

There are hosts that give you unlimited /32tb bandwidth monthly. That's quite generous

Livecasters with a large audience could fund development of multicast support for Owncast.