Netflix and Fill | Hacker News Reader

Netflix and Fill

techblog.netflix.com

111 pointssamber60 comments

> Because we can predict with high accuracy what our members will watch and what time of day they will watch it

Not only can they predict it, but I imagine to a large extent they control what people watch. Our usage often involves browsing the recommendations on the home page and picking something.

I wonder when (or if) the availability of content in a cache close to the viewer will play a role in the home page content? Obviously it already works in the other direction (cache what we think they'll want to watch), but do they penalize the ranking of a title (or category) because it hasn't been cached?

It'd be a useful feature for Netflix because they could cut down their bandwidth costs. It could even be useful for some users: those who could watch a movie at full quality from a cache but their ISP can't deliver a full quality non-cached title.

> Not only can they predict it, but I imagine to a large extent they control what people watch.

Oh man, new sinister revenue stream for Netflix incoming.

Next Google is going to realize that they can control which billboards or storefronts you drive by based on your Maps directions.

They kind of already steer people around with maps. Looking for lunch places near my office is terrible. It's mostly larger places. Good luck if your a mom&pop shop. I've also noticed that Youtube keeps trying to steer me towards videos I don't want to watch, and I don't really have a good way of telling it to stop.

> Not only can they predict it, but I imagine to a large extent they control what people watch.

This is similar to what happened with Google Play Music. They used to have a really nice feature that enabled you to browse through all genres and continue to drill down several levels into very specific sub genres. It was a great way to discover new music.

The problem was people were streaming way too much music that was not optimally cached and it was causing scalability issues for them. The solution? Remove the subgenre browse feature and transition to a radio station-based approach. You now have the choice of a limited set of predefined radio stations, each with a limited number of tracks, which I assume are all cached in an optimized way, but you cannot drill down nearly as far into the sub genres.

If you browse the google forums there have been thousands of comments about the reduced functionality over the past year or so.

To be fair, they paid good money for Songza's radio stations system. But yes, the browsability is awful. I don't mind the radio stations approach, but I can't even find good listings of what radio stations they provide... and when I do find a radio station I like, I find it has a rotation of like 12 songs.

This seems to be true of like at least 90% of music streaming services, just with varying numbers of songs per station. Like with slacker it was 50 songs, and then a small number of extras put in. Or amazon music playing the same song 2 or 3 times during the same listening session.

Just to echo Pxtl, but much more explicitly, I don't think this had anything to do with caches or scalability, it was because they bought Songza.

If you've any evidence, please do share and I'm happy to put my hands up and admit I'm wrong, but the announcement was that the change was supposed to be because they now had these amazing hand curated playlists.

Why not both? Maybe Sognza's playlists were a caching solution.

Occam's razor.

dlandis is suggesting a pretty over-complicated, anti-consumer reason as to why google started pushing curated lists.

Or you can buy into the incredibly simple "because google bought Sognza to compete with spotify's curated lists".

> Google looks forward to bringing back those genres/subgenres, and even microgenres, once they've had a chance to improve them and make them more scalable around the world.

That is a comment from a top contributor in the product forum. Thread is here https://productforums.google.com/forum/m/#!topic/play/i9meVy...

Wow, that thread is angrier than the Maps forum. Pretty impressive achievement on Google's part.

What's so bad about the maps forum?

So Netflix's utterly dire searching or access to their tags is actually a feature, not a bug?

I suspect their terrible interface is partially to hide how shallow the streaming catalog is.

No question about that whatsoever, IMO.

> Our usage often involves browsing the recommendations on the home page and picking something.

FWIW, my usage is quite different. I do browse the popular/recommended/etc., but only to add things to my list. When I want to watch something I usually select it from "my list".

I think that's a good point, and a weakness in my comment. I know everyone's watching habits differ, but I said "to a large extent they control what people watch" and then offered an anecdote to support it.

I doubt anyone outside Netflix knows what percentage browse the homepage, pick from their queue, or search for the video they end up watching. I do think it's safe to assume browsing is a popular choice, but it's not clear it's the dominant one. I'm basing this off the real-estate devoted to browsing on the home page. I think the UI would look different otherwise.

I also don't think "control" is the right word to have used, probably "strongly influence".

Caching is just an optimization, not a feature. If the content hasn't been cached yet, then there is a good chance that a caching is piggybacking on your transfer.

I was using "caching" to talk about the Open Connect Appliances and Netflix's proprietary CDN. It's probably the wrong word to use, given the differences in how the system works.

It doesn't sound like a customer watching a video stream causes that video to be copied to an OCA. The "Proactive Caching" section says they "want to make all of the OCA capacity available for content serving" and are trying to avoid read/write contention during peak hours.

It sounds like the system's working pretty well, the previous blog post says: "Globally, close to 90% of our traffic is delivered via direct connections between Open Connect and the residential Internet Service Providers (ISPs) our members use to access the internet."

Yeah, in my reading it doesn't seem like they cache on request. I don't think they said that they only cache some content though. They likely cache everything, but do so before making that content available to viewers.

At least for me, their predictions are terrible. I watched a couple historical documentary series on WWII and for the next couple months my recommendations were filled with Nazi conspiracy theory junk.

This exact same thing happened to me after watching a WWII documentary. I'm actually really into the WWII genre but their recommendations are terrible in quality particularly the ones from Smithsonian.

I semi-recently worked on a P2P video distribution system (not like BitTorrent, that's not really suited to sequential content) - not that different from Spotify's old system. One of the things we ended up doing to support new release binge-watching was to have a predictive consumption led pre-fetched content cache for network neighborhoods. It wasn't a change-the-world difference, but interestingly, a lot of people do watch the same video content over a period of not that many days.

I don't think Netflix generally pays for bw because of peering agreements?

As is often the case, reality is less sinister than fantasy: no, I can confidently assert that Netflix does not influence your catalog based on the state of the content cache. The system is entirely built the other way around: the cache will fill to meet expected demand, full stop.

This is one of my favorite blogs, because they go into satisfying detail about how their stuff works; it's always entertaining to see a big company write about how they accomplish what is it they do.

I wonder if they use HTTP or Bittorrent to serve out of S3; S3 can serve Bittorrent, and the protocol solves a lot of their 'fill hierarchy' optimization on its own reasonably well.

The title of this post shows such amazing self-confidence from Netflix in its own culture and tone of voice.

At 90% of companies the size of Netflix (and even some self-important, bloviated startups) an equivalent title would have been an absolute non-starter. As a result you get websites that all look the same and sound the same.

Kudos to Netflix, even in quite a small way, for having the confidence to do things differently.

I think it's more a reflection that the phrase "Netflix and chill" has become such a cultural touchstone―and a parody of itself―that even Netflix itself is in on the joke.

Sort of like what this did for "Thanks Obama": https://www.youtube.com/watch?v=uhY9Zxv1-oo

Kudos on the excellent blog title pun [0].

[0] https://en.wikipedia.org/wiki/Netflix_and_chill

Only on hacker news would you need to attach a reference to something most people should know..

Modern slang has no class. I'm pleased that I didn't know the term until I clicked the link.

Just modern slang eh?

You do realize your slang was seen the same way by your parents and their slang was seen the same way by your grandparents right?

Time changes, people don't.

Quite so, my dear fellow. It's ever so pleasant to cross paths with a fellow gentleman with class, unlike the rabble of today's youth. I do tip my hat to you dear sir.

Relevant xkcd: https://xkcd.com/1227/

Not everyone here is an US teenager.

I'm grateful for the reference (though after reading it, I'm having second thoughts).

Although teengager seems to be the goto term for when describing people who actually use new upcoming pop culture references and slang. It transcends into people well to their twenties and thirties.

Well yeah, that's the 18 to 34 demographic.

Not many people on here have much first-hand experience with Netflix and chill.

I remember calling it "watching movies".

and before that it was "Want to come up and see my etchings?"

Weird..

  $ curl https://techblog.netflix.com/2016/08/netflix-and-fill.html -vv
  *   Trying 2607:f8b0:4002:c06::d6...
  * Connected to techblog.netflix.com (2607:f8b0:4002:c06::d6) port 443 (#0)
  * Server aborted the SSL handshake
  * Closing connection 0
  curl: (35) Server aborted the SSL handshake

Fails from at least the 3 locations I just tried across the country.. Google outage?

Why do you use HTTPS? The link is HTTP, their blog doesn't support HTTPS.

Their blog is directing me to an HTTPS link when I go to "techblog.netflix.com". It does it even when I specify HTTP.

Do you have https everywhere installed too?

No. Firefox seems to be doing this on its own.

If you're running NoScript, check its settings. See: https://news.ycombinator.com/item?id=12256270#12257702

That did it, thanks. I would not have expected NoScript to be doing things like that.

Ah.. https everywhere seems to think it should support it.

Oddly have never had a problem with it until now.

This gets mentioned every time a blogpost from Netflix appears on HN. I've mentioned it too! https://news.ycombinator.com/item?id=12217900

According to kpcyrd, this is what happens: " I have the same issue, even though the link actually is http, my browser tries to open the https version, no matter what. It looks like https://netflix.com had hsts with includeSubdomains at one point in time which my browser has still cached. "

Which is why you should only enable HSTS if you plan on supporting it permanently.

"But there's no reason to not use HTTPS! What could go wrong?"

But, you know... we have to secure our blog posts from MitM...

No, you have to secure your blog posts because cookies against the top level domain can be in the request headers sent to the subdomain[1].

It's a bit bizarre you'd blame HTTPS, what is your alternative, don't use https?

[1] I say "can" because it depends on how the top level cookie is set, whether the browser respects the secure flag, and individual browser versions.

The google docs diagrams are being 403d for me

Edit: Working now :)

I've just notified the authors.

Looks like permissions on the diagrams hasn't been set properly. They need to make the link viewable by anyone who has it.

Weird, I'm getting a Firefox not secure connection error. Port 80 forwards to 443, and then 443 in Firefox throws throws the error. Curl just aborts on 443 also.

Netflix is cool enough to be in on the joke, but not cool enough to know that it's lame when a corporate entity has self-realization of the joke.

I think this is a bit different from when you get a mass email from a company along the lines of "We know you love to netflix and chill, so here's five new blockbuster hits we've added to our catalog".

This reads like something written by one engineer and reviewed by their boss. It's not nearly as bad as the regular /r/fellowkids stuff.

I think the mass email would be amusing if they only sent it to people who watch certain types of content at certain times of day.

Reply on news.ycombinator.com