Artwork Personalization at Netflix
medium.com
medium.com
That question seems revealing to me. Convincing me that a title is worth watching is good for business, I'm sure, but I want to decide what (and whether) to watch, not be convinced. Most titles are not worth watching.
I don't enjoy how relative and personal the recommendation system is right now, and I don't want more of it. The current recommendation system is showing me hundreds of titles that all say something like "97% match", plus or minus one percent, even though the majority of them are objectively crappy. It seems like the personalized ratings are all super inflated.
I want to know what other people thought and make my own decision, not be subject to a data collection and personalization system that filters everything through the eyes of a neural network that only knows what I've rated in the past.
It also feels like it's getting harder to branch out to genres I don't usually watch. The recommendations are giving me a metric ton of stuff that is similar (same genres, same actors) to whatever I rated highly, but not much that is dissimilar. It feels like I have to work harder now to find good stuff that is new or unusual.
Not to mention the surveillance chill of "how will watching this flick affect what the recommendation algorithm thinks of me." At least let me turn that shit off.
Maybe that's just context though, on say youtube where you have a billion vids and I'm just watching short music clips or programming vids, sure it may be helpful to show me stuff related to what I just watched. On Netflix just give me a directory and make it easy for me to browse, maybe show me the new releases etc. There's mostly no such things as 'similar' when it comes to feature length movies except for sequels and remakes.
Or I dunno, maybe they really just didn't have a single thing I wanted to watch, which is entirely possible.
Just do what everyone does and create a guilty pleasures profile? Bonus, your guilty pleasures account will get better guilty pleasure recommendations.
But as the the immediate point, guess I'm just paranoid. I want to watch movies without data nerds peering through my windows, is that so wrong? I would very much like to be excluded from their preferences dataset entirely, but as an alternative I'll settle for just not watching stuff.
When bae invites you over for surveillance and chill?
If they did that, you'd see just how small their catalog really is. For Netflix, the recommendation engine is vital camouflage for the fact that they just don't have very much content. Netflix have about 4,000 movies in their catalog, of which a substantial proportion are low-budget filler. Once you remove the obvious junk, you're left with a selection of movies that's no better than a Blockbuster store circa 2005.
Not to mention the fact that my wife and I use a single Netflix account. She watches things that I think are objectively garbage. I watch things she feels the same about. There are several things we enjoy watching together.
Just show me what you've added in the last month and make it easy for me to browse categories. That's all I want.
And how long is Kevin Hart's 3-year old stand up special going to be in the "recently added" section? It's been there for me for at least 8-9 months.
So I think your interest and their interests are aligned?
It’s your own responsibility to decide what to want and what not to want, but these days it’s pretty hard to fight the hundreds and hundreds of brightest minds that work full-time to convince you to like something. They know much more about the inner workings of your brain than you do, and you might not be equipped to tip the odds of manipulation in my favor.
I recently deleted Facebook and it was a real mind blower to realize how much they impacted my brain chemistry even though I didn’t feel like I used their service much. Since then I’m even less trusting of corporations armed with big data, big brains, big money and supposedly aligned incentives.
Whether the "learning" is being done by neural networks built in software or human wetware makes no difference. We're optimizing for the pervasiveness and uniformity of echo chambers in nearly every corner of our lives. Those echo chambers are largely responsible for greater polarization of political views, lack of ideological diversity, the snowflake generation, and a host of other problems that seem to be getting worse by the day.
It's because, instead of informing about their offer honestly and letting people decide, people figured out they have to convince buyers. Given that this is mostly a highly-competitive zero-sum game (for a given established market segment you're in, you're competing with others for limited amount of potential customers), I suppose me complaining about this is like complaining that the water is wet, or that speed of light is too slow.
But pause for a minute and consider, how so many of this bullshit - including "greater polarization of political views, lack of ideological diversity, the snowflake generation, and a host of other problems that seem to be getting worse by the day" - all boil down to individual entrepreneurs who engage in convincing to bump their income a tiny little bit each step of the way.
> How do we convince you that a title is worth watching?
Here's another quote that balances that one a bit:
> We also carefully determine the label for each observation by looking at the quality of engagement to avoid learning a model that recommends “clickbait” images: ones that entice a member to start playing but ultimately result in low-quality engagement.
I read that as saying that they want to avoid tricking you into watching stuff that you won't like. Certainly an interesting problem.
I agree with your points about how hard it feels to find new, interesting stuff on Netflix sometimes. The browsing interface feels like it's set up to make low effort discovery easy, but not to find content that's really great and unusual.
I often end up using a third-party site like Rotten Tomatoes to help me find something interesting. Perhaps that's because I trust independent reviews more than Netflix recommendations, especially now that Netflix features their own content so prominently.
It's like Netflix has (at least) two different population of users - those like us, who would like a catalogue of legally streamable good movies to choose from, and the other population for whom Netflix basically replaces TV, i.e. people who don't really care what exactly they watch, and thus are open to recommendations.
I suppose it's fair if they want to target that other population. Maybe they're majority. For me, their recommendation is only an utter annoyance, and it lost any value the moment they switched from objective ratings to "percent match".
I believe Netflix moved to the current model because they don't care about you spending your time well and in addition need to fudge their catalog. It's ultimately wasting user's time. Before they had the new match percentage it regularly would suggest movies that they themselves predicted I'd hate. WTF?!
At this point my favorite movie platform is mubi. Just 30 well selected movies. I can quickly tell what's new to their catalog and also if I should just close the app and do something else.
Even basic parts of the UI like "My List" or "Continue watching" aren't reliably available in the same places.
It is somewhat funny to see, for many years now, recommendation and ratings take such a heavy back seat when they were previously one of Netflix's most famous features.
You're gonna get higher click through in the short term but lower satisfaction in the longer term. This "up-and-to-the-right" disease will erode your credibility by not treating artwork as canon in subtle ways and alienate your audience who are disappointed Good Will Hunting, in this example is more drama/romance than comedy, and that Robin Williams, (usually) a comedian, is the most serious character (he graduated from Juliard).
tl;dr Long time personalization product manager here, don't do this: it will hurt you
This is like having a vinyl music collection made up of hundreds of items where the covers change randomly. Please Netflix, stop doing this.
I actually think this algorithmic discussion is really fascinating.
Haven't seen on Netflix, using your (sub)account. This breaks in an annoying way when you saw the artwork previously at your friend's party, or on your spouse's subaccount, etc., and are now skimming the catalogue for the movie (maybe you don't remember the exact title, but know it's in a particular genre list).
It is based on their engagement score for the title. With this change, folks are consuming more content, so I don’t think it is correct saying this will hurt them based on your own negative reaction.
5 years ago I wrote the recommendation system that Netflix uses (and has degraded since then). One major problem is in the past certain senior Netflix managers are only interested in self promotion (I would hate to extrapolate to the current ones - even if the extrapolation is reasonable). A/B tests are a perfect device for this. What is a better recommender? They were not interested in improving the product. It is easier to win by politics/lying/obfuscation/omission/plagiarism then come up with better ideas. If the company goes down they move on with a good resume and the games they played (for instance USPTO fraud), are hidden.
Scroll down for my comments here: https://www.reddit.com/r/MachineLearning/comments/6xiwr4/d_w...
Some companies are like this. For instance the "Netflix prize team" at Verizon/Yahoo refuse to share recommendations data with other teams leaving them nothing to do. They work in a bubble and will actively try and remove anyone who might be a competitor.
It's sad that Netflix decided to pursue this work (+ other even more brain-dead projects like rewriting the command-line parser, "switching to OO" - examples of Xavier's initiatives). They could of 5 years ago pushed the beta system that has 40% extra performance. (I'm currently at a factor of 3 better performance).
For example I watched some shitty movie recommended by Netflix and I feel less satisfied with the service than if I didn't watch it at all.
When I create such and such a piece as an artist I want to present it a certain way, even down to the marketing material--I want it all to express the correct vision--i.e. there is one 'correct' promotional image for it.
Marketers, contrarily strictly have the utilitarian principle of conversion in mind--they want to manipulate my vision to make it a better fit for as wide an audience as possible and in so doing diminish and cheapen the art to target certain subsets or groups who may not actually fit the intended audience when you consider the work as a whole. Art is reduced to a profit making instrument.
That's precisely what this method attempts--cast a wider net by deceiving the fish and ignoring the wishes of the fisherman, who, only wanting trout, now gets the whole biosphere of the sea in his boat.
Its not very honest, and will probably make people upset unless the content based recommendation algorithm aligns perfectly with the image focused one (i.e. the image is only manipulated to fit your taste after it's ascertained the content as a whole and on its own fits your taste, and not only a subset of that content, i.e. one or two brief romantic scenes in a movie that has as its actual subject matter something entirely unromantic).
Also, these "versions" of product are quite common in different forms of art. Songs have radio versions, album versions, remixes and simply alternate releases. Whole albums have different masters (released for different medium and audience). Movies have director's cuts, movie cuts, TV adaptations (where they sometime get serialized). Books composed from chapters printed periodically in magazines, with author responding to reader's feedback (which was much more common in 19th century, but still).
For me, I think it basically just reduces my engagement. If my recommendations are filled with things I've already decided I don't want to see, I'll just go do something else.
What the heck happened to their UI? It used to be quick, and reasonably useful. At this point, I'd rather go to Amazon and try to guess which search result (and on which page!) is the video I've already purchased.
What? If I go over to a friend's house, I don't think of myself as using a different product. I don't want to use a different product. This advice seems to go against everything I've heard about brands and recognition.
My perception is, they're a big enough brand now that they don't have to care anymore.
You can deal it, but everyone should be aware of this before doing the naive thing of just throwing bandits at things. Sometimes some regret is worth the plain validity of an RCT, and sometimes not.
My wife and I own two gyms and payroll is our largest expense by multiple orders of magnitude. Every business I've ever looked at or been involved in, payroll has been the largest expense, often by one order of magnitude or more. The fact that licensing still beats out Netflix's payroll is insane.
Different businesses have different cost structures.
I used to run a hosting company and as we scaled up our bandwidth and infra costs eclipsed our payroll costs.
In many retail businesses rent and advertising eclipses payroll.
With regards to Netflix in particular, I think it makes sense that licensing costs are higher than payroll costs.
How about, instead of overpersonalizing everything, you say "we recommend X because you liked Y and people like you also like X."
I wish sites were more up-front about it. In many places, I get a recommendation and I wonder, "why on Earth did the site just suggest that?".
Steam does a little bit of what you suggest for games. When I browse their recommendation queue for me and I see a weird title, I can look down and see a little text that say something like "This game is suggested because it's new on Steam", or "because you've played similar games", or "because you've played games tagged XYZ", etc. It makes me feel that they don't pull the queue out of their collective ass.
Of course it is nice to listen to music we love and to see movies that fit our past viewing experience, but what about putting some plain randomness in there too. Wikipedia has a "random article" button. The online content world is not random enough.
About the article:
Very interesting idea, and the Replay is a fascinating and in hindsight obvious way of being able to quickly test out new approaches without waiting for the data collection.
Screaming into the void:
If you're able to do all this, why do you still write "NEW EPISODES" on the thumbnails of shows you know I've already watched? Why do I have broken images constantly in my android app? Why can I not turn off autoplay of trailers (meaning if I don't want to have things spoiled I need to keep flicking around)? Why do you suggest I watch things I've 1. Already seen and 2. Told you I hated?
Since the recommended titles are clearly very focused on Netflix production, I just ignore any recommendations altogether and just browse all new arrivals every week...
I do the same only to find that there's nothing interesting there and watch Arrested Development again.
Also I am not sure if I want a different "Poster" for "My Cousin Vinny". Ever.