Hacker News Reader

This comments on why bittorrent as is isn't used for live streaming, not why P2P shouldn't be used for live streaming

> Latency matters. In bit torrent if the peer goes away, no big deal, just try again in 5 minutes with another peer, you are downloading in random order, who cares if one piecs is delayed 5 minutes. In a live stream your app is broken if it cuts out for 5 minutes.

First of all, BitTorrent clients do not download in random order or wait 5 minutes. They usually download the rarest block first, but can do whatever they want, whenever they want.

Second, standard HLS sets a nominal segment size of 6 seconds (some implementations will go as high as 10 seconds), and a client will usually cache multiple segments before playing (e.g., 3). This mean that you have 18 seconds before a segment becomes critical.

This is not a difficult thing for a P2P network to handle. You'd adapt things to introduce timing information and manage number of hops, but each client can maintain a connection to a number of other clients and have sufficient capacity to fill a segment if a connection fails. Various strategies could be used to distribute load while avoiding latency penalties.

Low-latency HLS uses much smaller segments and would be more demanding, but isn't impossible to manage.

> BitTorrent punishes people who dont contribute

Private communities punish this behavior, BitTorrent clients do not. Most new downloads will appear as freeriders for a long time, and only over long periods far exceeding the download time will enough compatible seeding opportunities arise for them to contribute in any substantial way.

The network does not need everyone to seed, it only needs enough people to seed.

I have a relatively slow ADSL connection, it's not unusual for me to be able to download 100% of the file at say 95% the theoretical rate without uploading anything. If the network has enough upload capacity to do this, does it really need my upload? (Note my client is still there if somebody needs a rare block)

I remember some Bittorrent networks circa 2005 or so which tried to monitor you and punish you for not contributing and this was a disaster for me since my upload is necessarily a small fraction of my download. What I found is that kind of network seemed to perform poorly even when I had a good bidirectional connection. As I saw it, people who have the ability to upload more than they download are a resource that that kind of network can't exploit if everybody is forced to upload as much as they download.

You are not required to seed immediately, nor even the same asset. Unless the rules have changed, you would usually just be required to maintain a particular minimum seed lifetime ratio that is easily achieved by idle seeding a library.

The point is to ensure network health with a metric that is simple to understand and verify: that you have been a productive. If you aren't seeding, someone else has to pick up the slack and the network didn't benefit from you obtaining the blocks.

The community itself benefits by giving members a guarantee that stuff there is available with abundant bandwidth, instead of relying purely on the unpaid goodwill of others.

You can't upload much of what's rarely downloaded. The usual way to help the network is to download something popular that you don't need and to seed it. This removes some strain from big seeders and helps them seed other things you don't have.

But personally I believe you're not obliged to anything in torrents, especially with adsl. We got you covered with our multi-hundred u/d ratios. Everyone participating honestly and comfortably is enough, even if that means no upload. Just don't turn it off out of unnecessary principle after downloading.

> First of all, BitTorrent clients do not download in random order or wait 5 minutes. They usually download the rarest block first, but can do whatever they want, whenever they want.

The problem here is that BT works so well because the clients default to "good behavior" (prioritize rare pieces first) and discourages "bad behavior" (leeching/no upload).

This tilts the balance on the whole enough to maintain the health of the network. If you change these, you'd need to introduce other mechanisms to preserve the network incentives.

With streaming, the service provider is generally also controlling the clients, so they should be much better off when it comes to client behavior.

In case of bring-your-own-client, the incentives are exactly the same: Clients would likely default to good behavior as network health equals user experience, and exactly like BitTorrent there will be neither punishment nor need for it if some clients disobey.

In video streaming the client really controls everything...unless you have your own client and customize its behavior.

You should see how people try and get HLS to pick a stream. With the default players it's not possible - the client does it.

Yes that's the intended behavior of HLS - the content provider advertises which streams are available and at what bitrates, and the client picks which to use based on the current stream performance and its own capability.

The server can control the stream by advertising a customized manifest to individual clients, although it's a bit silly. HLS is designed to be extremely easy to load distribute and throw CDN/cache proxies in front, and it's a bad sad that content providers are this bad at load management. :/

Either way, the assumption here is that you would swap out the client doing HLS with a client designed to use a P2P protocol, including the seeding portion and network health management.

> exactly like BitTorrent there will be neither punishment nor need for it if some clients disobey.

"Punishment" (tit-for-tat algorithm) is one of the defining features of bit torrent, especially in comparison to what came before it.

This is not a "punishment", and what clients to differs greatly.

The original spec for the original client allocated a portion of its bandwidth to random peers instead of known-good/preferred peers, so if you had no chunks you were basically bandwidth and/or peer restricted.

If you take the arch linux ISO right now and put it into aria2c to be a new, unknown client with no data to offer, you'll find that while it takes a few seconds to join the network, fetch metadata and connect to peers, you'll quickly saturate your connection completely without ever uploading a single byte.

If you wanted, a streaming network could use direct access or low-hop access as seeding incentive - seed to get slightly lower content latency. When the streaming client is controlled by the content provider, seeding is easily forced and topology could be controlled centrally.

>This tilts the balance on the whole enough to maintain the health of the network. If you change these, you'd need to introduce other mechanisms to preserve the network incentives.

This is the real key point here. In the P2P wars, only bittorrent was the winner (though the old networks still live on and you can find interesting stuff in them...) - the timeless lesson is leeches need to be getting the short end of the stick, people who give back, the prize bonus. It's a fundamental characteristic of human nature here, tragedy of the common leech or something like that

ish? it's pretty stupid that in the age of seedboxes with gigabit we still prioritize that. The only way to be a seed these days is to get the release microseconds after it gets released, and download it before everyone else gets their hands on it, so you can upload to them. Everyone else then has to download from you, but since everyone that's more than slightly involved has a seedbox with symmetric gigabit, the issue of leeches just isn't the same as it was before in the dial up and DSL era. That the culture hasn't been able to evolve past that, and hangs on to such outdated notions is disappointing. I'm not saying people shouldn't be incentivized to give back. Leeches suck. But in this land of plenty, theres this problem of being unable to seed because there's just too many seeders trying to seed.

Nobody who does live streaming should set their gop to 6 seconds. That's way too big. I set ours to 2 seconds, which lets mobile adjust more quickly to their bandwidth limits.

And as a note, video segments for live are usually set to no cache, as are vod segments. The CDN does the caching. The client should keep segments around if you're doing rewind, but that's a client thing.

I don't think people are appreciating the nuance of what you're saying. Most of what you are saying isn't accurate for netflix style streaming, which would actually be more aptly called "video on demand", but is very applicable to "live streaming" in the sense of live sporting events or news broadcasts.

Video-on-demand is perfectly implementable on top of BitTorrent. As you say, there are some latency pitfalls you'll have to avoid, but that's nothing you can't hack yourself out of.

Livestreaming is a different beast. As you say, the problem with livestreaming is that everyone needs the same content at the same time. If I spend 200ms downloading the next 500ms worth of content, then there's nobody to share it with, they all spent the 200ms doing the same. BitTorrent relies on the time shift that is allowed between me downloading the content and you requesting it. If you request it before I've got it, well I can't fulfil that request, only the guy I intend to get it from can.

If you wanted to implement something like that, you would probably pick a tree of seeders where the protocol would pick a subset of trusted nodes that it would upload the content to before then allowing them to seed it, and the have them doing the same recursively.

That would obviously introduce a bunch of complexity and latency, and would very much not be BitTorrent anymore.

I don't think he's saying it needs to be BitTorrent, just applying some principles from it.

For example, say you have a cluster of people on the call in the US and another cluster in the UK. Ping times are 100ms or more across the ocean, there will be some random packets lost, but ping times within the UK are around 15ms max. By working co-operatively and sharing among themselves the clients in one cluster can fill in missing packets from a different cluster far quicker than the requesting them from the originating host.

In general, the ability to request missing packets from a more local source should be able to improve overall video call quality. It still might be "too late", because for minimal latency, you might choose to use packets as soon as they arrive and maybe even treat out-of-order packets as missing, and just display a blockier video instead, but if the clients can tolerate a little more latency (maybe a tunable setting, like 50ms more than the best case) then it should in theory work better than current systems.

I've been mulling over some of these ideas myself in the past, but it's never been high enough on my TODO list to try anything out.

> By working co-operatively and sharing among themselves the clients in one cluster can fill in missing packets from a different cluster far quicker than the requesting them from the originating host.

That's only true if you assume the nodes operate sequentially, which is not given. If the nodes operate independently from one another (which they would, being non-cooperating) they'd all get a response in ~100ms (computation and signaling time is negligible here), which is faster than they could get it cooperatively, even if we assume perfect cooperation (100ms for the first local node + 15ms from there). It's parallelism. Doing less work might seem theoretically nice, but if you have the capacity to do the same work twice simultaneously you avoid the synchronization.

Basically, it falls somewhere in my loose "tree based system" sketch. In this case the "trusted" nodes would be picked based on ping time clustering, but the basic sketch that you pick a subset of nodes to be your local nodes and then let that structure recursively play out is the same.

The problem you run into is latency. There's no good way to pick a global latency figure for the whole network, since it varies by how deep into the tree you are. As the tree grows deeper, you end up having to retune the delay. The only other option is to grow in width at which point you've just created a another linear growth problem, albeit with a lower slope.

I generally agree, but once you do that you lose most of the properties that make bit torrent so effective.

E.g. if you arrange the network into a tree like that, you need to make sure all nodes are matched appropriately in terms of bandwidth, latency, geography, number of connected nodes. Now you have to somehow the network topology stays good in face of churn and bad peers. Suddenly everything is complicated and not looking very p2p.

Maybe dufferent protocols are possible to manage that, but i think there is a reason why p2p protocols kind of didn't develop much beyond bit torrent.

If I remember correctly, PopcornTime was able to stream via BT. Your claims are mostly correct, but I think some compromises can be made to make BT clients streaming proof. For example:

1. locally-randomizing segments download order

2. Create a larger buffer

3. Prioritize parts coming from slower connections

The main difference is liveness of the stream. Live streams are much much more difficult and much less forgiving.

This is still just streaming a static file though. Adjusting which segment to get will work, buffering will work, and people don't mind their movie starting a few seconds after they press play.

If I'm streaming live, I need the frame immediately, and it doesn't help much to get later frames after the frame I'm missing.

what if everyone agrees on a 10s delay?

The issue is everyone is watching the same part of the file at the same time. Offsetting by 10 seconds does not change that.

The time offset impedes the ability of viewers to interact with the streamer via chat, which for many people (incl. myself) is the whole reason to watch live instead of a recording.

For pseudo-live streams such as sports events, that would be totally fine. People can have slightly out of sync streams, delayed by various amounts.

But you can't live stream a conversation with someone if you have a 10s delay.

Latency is actually a pretty big deal for live sports events.

No one wants to hear the cheering start at the neighbor's house ten seconds before you get to see the team score a goal.

And yet, that is exactly what we have!

Define "live".

Live streaming is, by nature, a "one-to-many" distribution model, where content flows from a single source to many viewers in real time.

BT, on the other hand, is fundamentally designed for "many-to-many" distribution, where peers share pieces of content with each other over time. This isn't just a question of tweaking the protocol—it's a fundamentally different problem. Unless you're willing to compromise on the immediacy of the stream, using BT for true live streaming isn't really a good fit.

I remember testing popcorn time and other bittorrent streaming tools back in the days. They worked "OK". Yes, you don't get the netflix experience. But on popular titles you get "good enough" streaming experience. You have to wait like 30 secs to get started.

Part of this was the fact you needed a pretty decent connection speed and that the files themselves were extremely compressed

I am not convinced about the random order stuff. If most people will stream from the start then the start will be more seeded? So it's all good.

And the order is requested by the client, and there are clients that go in the sequential order, like Deluge

The benefit of a random order is that it forces you to actually keep all the packets, which makes upload more likely. Streaming lets you get away with not storing the whole file, which makes bad actors more likely.

And, sure, some BT clients can stream the data, but what the default is makes a huge difference.

People will shut off the moment they get the full file. The randomness means that the last packet is as likely to exist as the first.

Would you want to watch the beginning of something that didn’t have an ending? How frustrating would that be?

> People will shut off the moment they get the full file.

Perhaps but the time spent downloading it is also time spent uploading some of the file, so there's still some benefit. By having it in random order, you more evenly distribute the people with access to different parts of the file.

With streaming, if everyone downloads the same blocks at the same time, "bad actors" can dump all data they already watched to save disk space, harming potential peers that are watching slightly behind.

Proof of work has problems with this because you (Mallory) can be paid to be the tertiary durable store for a file and secretly fetch the file from Alice or Bob when asked to prove you have the files. And even if you do something like use a different cypher for each copy the fact that the data is often meant to be public means one could work out the cypher given Alice and Bob and then dump your copy once you have done so.

Unless you use public key cryptography, which is so expensive that nobody actually uses it for arbitrarily large inputs.

If we are talking about a live stream (and not a stream of a static file), having the start be more seeded is useless.

mpeg stream / TS filetype / DVB-T/C/S broadcast IS static file, all 3 is same format, this format deals with every point you made... download specs and educate yourself.

same with streaming audio, chunk IS static file, so every phone call you made last 30 years is static file.

and there is no reason for bittorrent to not being able to send/download 3000 TS chunks / static files for 3 hour movie in sequential order.

and no reason for your MPV/VLC/PotPlayer to not render that in sequential order.

even when you have only first 2 pieces.

The word "file" is doing a lot of heavy lifting here.

what is difference what kind of data chunk it is? 0 difference. or for university educated - he introduced file as a helpful abstraction, so i work with that abstraction, if you say it is not good abstraction then tell him. calling ts file a file is absolutely correct in any sense of word. just to be thorough. in philosophical debate we are having in computer science. ( yes toxic sarcasm )

that one post is more to the topic of OP is asking for, than 90% of comments here.

again NEWS, movies, comedy, trumps tarrifs, are streamed digitally to bilions of people over dvb-t/c/s every day. if how are bits ordered/chunked is important for you that much, that this already working system is not good for you in that sense ? makes no sense.

or explain little more, one sentence explaining whole world is k12 like. or 42 for book readers.

I'm using "static file" to mean something pre-recorded that even if users will view in order, they will likely start at different times, so different users will be viewing different parts of it.

In contrast to a live stream where everyone is viewing the same part at the same time, and once that part passes nobody is likely to view the old part ever again.

This makes a big difference in terms of network design.

yes so makes no difference if it is file "without end" (pipe essentially) or bunch of TS files.

The issue is not how the bits are divided into packets (or "files") but how those packets are distributed/used.

Obviously at the end of the day its a string of bytes, like everything is, the difference i'm trying to get at is differences in how the data is used and requested.

Its more a social difference not a technical one.

If everybody starts at the start then it will be very poorly seeded when everybody wants it, before the being well seeded right when nobody needs it.

"well seeded" means what exactly?

if i can send 2 copies of piece to 2 people immediately as i got it, then if my download takes 20 ms and sending it another 20 ms is it "well seeded" for those 3 people after 50 ms? or after how much time it is "well seeded" ?

A precise answer to that question entails more math than I'm willing to do right here in the middle of my Easter holiday. You should understand my argument more as a sketch of proof.

That being said I have a small correction. If you want to stream to two peers (that is you have a network with 3 fully connected nodes, one being the originator of the stream) and the link latency for all links is 20ms, then your lowest latency path to each node is exactly 20ms, as the originating node could simply send the content to each client.

The unfortunate realization is that 20ms is then also the optimal stream delay, assuming you care about the shortest possible latency for your live streaming service. The clients will therefore end up with the content exactly when they were supposed to show it, and therefore they would have no time to forward it to anyone else, lest that downstream would get the content AFTER they were supposed to show it.

Downloading and reuploading part of a file in 50 ms is optimistic and that is still only three people when serious live-streaming platforms have to deal with thousands of viewers routinely and millions every so often.

That's only true if everyone starts at the very same moment in time. Of course that is not reality, and it works out well in practice.

yes you are correct, BitTorrent - sequential download also works exactly like that.

people seem to have need for 0ms nano ultra low latency streams for watching movies,... they are insane. they want to be extraordinary high speed traders but with movies not stocks. insane

Just for shits and giggles, I wonder if a protocol similar to HLS, where each peer is assigned a segment to transcode / cache, would work for best cases. If a peer drops, that segment is assigned to another peer. Or if there are, say, 5 peers, double up on the next segment to increase odds it will be done.

Just a mental curiosity is all.

> Part of the reason bit torrent works really well is that the file is downloaded in random order.

That's basically true for one client (transmission) - who specifically refuses to allow linear ordering. Most clients implement this.

To enable it, its about a 3 sloc change.

I hate clients that dont work for the user.

>That's basically true for one client (transmission) - who specifically refuses to allow linear ordering. Most clients implement this.

Transmission does allow you to set a particular file (let's say the first file in a series) as 'high priority' though so it's not like they don't allow any change to the behavior

good for them. they specifically refuse to implement this feature even though its a tiny change for the 'good of the ecosystem' or something stupid.