mranton

via Dan Rayburn:

"""

My bullet point high-level thoughts on the Super Bowl stream:

- Numbers aren’t out yet, but I would expect an AMA number of under 9 million (2023 was 7 million AMA)

- While some were debating online if the Paramount+ stream was in 4K, it wasn’t. Their stream was 1080P 60fps true PQ HDR, had no color space conversions and was encoded for six bitrates maxing out at 12Mbps.

- Depending on the device and platform, I saw latency from 2 seconds to 60 seconds behind the TV broadcast feed

- HDR on Parmaount+ looked great across all devices I streamed on, and the upscaled 4K/HDR/5.1 stream on YouTube TV looked great as well

- Some users, including myself, did experience some errors or crashing of the stream, which started before kickoff and was largely mitigated before it could impact a larger portion of users. I put the impact to 1% or less of the total streams delivered.

- One media outlet reported, "The streaming platform seems to be struggling with all the Super Bowl traffic,” which was inaccurate. Paramount was prepared for the viewership they received, and there was no problem with the volume of users.

"""

bunny.net is a Slovenian global company with no affiliation with Russia.

The more CDN providers, the more choices you have for less money. If you are using Vultr as a hosting provider, it's much easier to enable/disable CDN out of the control panel.

everything works in our sandbox. We need a couple of customers to collect real data. I'll definitely going to post again with these numbers.

>Or, the site owner could manipulate the coordination JavaScript to report the downloaded file as invalid/corrupt/tampered with, and then just not re-request the file (and there's nothing you can do to stop them). So in that case, the site owner is managing to not pay for the service (assuming you don't bill for downloads that were reported as invalid).

This fraud is possible. We collect logs from three sources:

- the Balancer (belong to Farba)

- the Peer

- the User

We can analyze every single case of suspicious behavior.

for unpopular content, we will deploy a limited number of servers (belong to Farba) with a performant disk subsystem. A web cache storage is a separate option in every CDN: if you want your files always in "hot" cache, you have to pay extra.

to answer this question, I will write another article how we distribute cache among the Peers. Long story short:

- we divide web cache in buckets

- these buckets have different sizes

- we know a list of files in every bucket

- we know what buckets every Peer has

A customer can switch on/off regions. Also, they can set up a cap for every region. You can do it with regular CDNs as well, it's just a setting.

>The Red Swoosh peercasting tool is a _browser_extension_ that caches data, reflecting and sharing files delivered through the "Swoosh network"

I don't know how it works in details, but it looks like you need to install a browser plugin to enable the p2p network here.

Our solution works transparently for the end user.

Thank you, @jeroenhd, you got it right.

>now your battery-starved mobile phone is validating cryptographic signatures

with a native app, it costs nothing. Actually, the main power consumer is the phone screen.

>who's responsible for illegal content hosted on servers

this is a very important question. We are going to run AI models against all content we host on Peers. Right now, we don't accept third-parties and "fake" all Peers ourself. Want to ask lawyers what is the best way to handle this case.

>Hopefully you at least make sure to try and return a single host for all the objects in a request.

our goal is to saturate the _User's_ network. Now it's hard to say how many Peers we need to meet this requirement. It looks like 2-3 Peers are enough.

Also, when you have 50 Peers, you have to do 50 DNS/SSL requests in _parallel_. With current hardware, it's not a big deal.

>Your example servers have 16-32 GB of memory, which is basically nothing

we just need to add more Peers and evenly distribute the workload.

> what kind of cache hit rate are you expecting at the peer?

I don't know. We have too many moving parts right now and can't predict anything.

I'm collecting use cases when Cloudflare stopped service for a "non-web content" rule violation:

https://news.ycombinator.com/item?id=34639212

correct

IPFS is very slow. It's like a cold storage. Not good for web-cache at all. Though Cloudlare provides gateways for IPFS, and again, we have a CDN layer here. It would be nice to see IPFS performance numbers to compare.

>a CDN isn't just about bandwidth, it is about _disk_

The disk performance - is the weakest part of the Peer. It is a part you can't guarantee any QoS on a cheap server. The idea is to keep popular cache on Peers in memory. Everything else we will host from a limited number of regular servers (belong to Farba), with high-performance disk subsystem.

>Cache management (eviction, invalidation, etc) is done where?

yes. You can invalidate either a single URL or a folder with a wildcard

>you are actually doubling your origin traffic just to push to the cache after serving the original client request

this is something we can proxy and do only one request per file. Current implementation was done for simplicity and reliability.

>but I don't see how this can work. Are they going to rewrite the object in a way that doesn't require the host certificate?

you can easily check this up. Please open the network console on our demo-page and see how it works:

- we have two different certificates (one for the main site, for for the Peer)

- everything works flawlessly

>1. Performance.

also please note:

if you have 1 image on the page - we have to do 2 requests (1 metadata, 1 download)

if you have 50 images on the page - we have to do 51 requests only (1 metadata for all images, 50 downloads)

kind of. They are WebRTC-based CDN.

>Something doesn't add up here. They're actively telling third parties to lease cloud servers with large amounts of free bandwidth:

Bare-metal servers are much better is our case. Please note, bandwidth is not free. You can follow the links and see the actual price for it. If LeaseWeb tells you: 1 Gbps dedicated/unlimited per $143/month, it means you can use it 24x7.

>- They are paying the third party less than what it costs to rent the server

we don't know prices in all datacenters all over the world. That's why the _Peer_ sets the price for his service.

>1. Performance. In terms of latency, this is adding an extra round-trip to each page request (to get the metadata). It's also likely going to load different resources from different servers (hostname, IP) preventing connection reuse. I guess they can't even start fetching data until the full page has loaded?

fair point. We use same technique as lazy-load (you have to wait the full page has loaded).

>3. Economics. If running peer servers really was profitable

Uber doesn't drive taxis himself, airBnB doesn't own property. This is how Uber-economy works. We believe it works in the CDN industry as well. We just don't want to be another CDN provider.

>How well does this scale? Are peers all necessarily running in major cloud environments?

The network scales very well. The network is divided into multiple GeoZones. We can add remove these zones easily. GeoZones are not connected to each other, so there is no single point of failure. Actually the major cloud envs are the most expensive possible variants for Peers. Bare metal servers are much cheaper in our case.

>Do you imagine seeing peers in the form of smaller players with beefy home setups?

Home internet has a couple of disadvantages: - traffic is not "unlimited". Xfinity doesn't allow you more than 1 Tb in and out traffic per month - home internet is asymmetrical: you could have 1 Gbps in and only 5 Mbps out.

right now no, but we have an idea to opensource everything.