Comment by grecy | Hacker News Reader

Comment by grecy | Hacker News Reader

Parent Full thread

I work at an ISP with local caches from a few of the big players.

We've been contemplating getting a NetFlix OpenConnect device, but as yet it doesn't make sense for us.

The way it works is that once per day, it will sync the ENTIRE NetFlix catalogue to the OpenConnect device, which is an enormous amount of traffic (for us). Even if it's a movie or TV series none of our customers have ever watched, it still gets synced.

Because we only have ~25k internet subscribers, it would actually be more traffic to sync the OpenConnect device daily than we're currently pulling from NetFlix.

Obviously this would be different with vastly more customers, and we're watching the numbers as more and more of our internet customers subscribe to NetFlix.

We wish NetFlix's caching worked like some of the other big players, in that "popular" content is cached, as is anything watched by someone. If an asset is not popular and never watched, it will never move into our network.

(From the Netflix PDF)

Each appliance stores a portion of the catalog, which in general is less than the complete content library for a given region. Popularity changes, new titles being added to the service, and re-encoded movies are all part of the up to 7.5TB of nightly updates each cache must download to remain current. We recommend setting up a download window during off-peak hours when there is sufficient free capacity.

You can work with Netflix on this. Your peak usage time as an ISP should be 9 AM to 5 PM or 6 PM to 11 PM depending on if you serve businesses or residential. You then schedule the sync to be at 3 or 4 AM when normal traffic is at the lowest, resulting in minimal impact.

You can even get a cross connect (I believe Netflix will even reimburse your side of the CC fees, but don't quote me on that) to a Netflix hub so loading will not hit your 95th billing with your upstreams.

Yes, we would schedule it in the very early hours of the morning.

We are much, much too remote to get any kind of cross connect. We only have one option - our ~3000km of fiber.

I presume they are syncing via something like rsync -- so besides metadata, you wouldn't be re-downloading 100TB's of video content every day.

But yes, you are right, OpenConnect targets the large ISP's (mainly the Big5)

7.5 terabytes / 10 gigabits/second in minutes - 100 minutes - you are right - that is a pretty heavy sync.

and because of our remoteness, we don't have nearly that fast of a backhaul to the outside world.

Reply on news.ycombinator.com