Netflix Recommendations: Beyond the 5 stars
techblog.netflix.com
techblog.netflix.com
The same idea is been commercialised by Kaggle (http://www.kaggle.com/) but there are several issues. Of course there is less up-take as the idea is no longer novel and the prizes are less. More than that, I think people are realising that winner-takes-all sucks, and the winning entries tend to combine so many different techniques that, as Netflix found, putting them into production is difficult. There is some interesting work on a better model here: http://arxiv.org/abs/1111.2664
What I want is a recommendation of what I'll probably like. It is absolutely irrelevant if mid-range movies sore a 2 or a 3. If Netflix can pick out a list of movies that I would rate a 5 (and maybe even 4), they've got the holy grail. Nothing else matters. So why optimize your algorithm to capture the 2s and 3s as well?
For bonus point, it might be nice to be able to pick out the real dogs. If it could warn me that I'm about to rent a 1 or 2, that would be cool. But it doesn't matter if they can tell me which of 1 or 2 it is. The precision is irrelevant, just tell me I won't like it.
(If I've said this once, I've said it a hundred times. But I guess I'll keep on like a broken record as long as Netflix keeps trumpeting what an achievement the Prize's algorithm was.)
In the OPs article they mention they monitor if a movie is watched to completion, which gives them a much better metric to optimise. The other issue is that this is really a sequential decision making problem. Recommending a movie has an opportunity cost -- there are other movies you don't recommend -- and the recommendation is an ongoing process, so it is probably best to spend some time exploring the user's taste on the assumption this will let you make better recommendations in the future. Accounting for these issues is much harder in a competition format.
It seems to me that the rating algorithm is a bit sensitive. Also, I wonder if Netflix has considered giving people they option of multiple "personalities" for the purpose of suggestions, queues, etc. I bet if they offered this for something like $3/mo more, people would pay for it.
It looks like they explicitly remove that for streaming customers. I have no idea why.
Personalisation is weird when they're not even dealing with people but households.
Householdalisation?
It's brand new and is my first web app so I'd love any feedback , good or bad.
"You watched and liked XYZ, your 38 year old wife hates ABC, and you've got 3 daughters 8/10/12? They'll probably like these shows ...."
It seems they missed out on years of awesome data by not allowing for segmenting by persona from the early days.
Their FAQ mentions it. But when I asked their support line was told it's only available to subscribers of the DVD service.
"Ah, there's my kids Dora selections again.. where's my recommendations?"
I agree, it does complicate things some, but consider
A) not every account would need it. B) not everyone offered it would use it C) the accounts that used it would probably be a bit savvier to start with D) they'd get a lot of extra valuable data, not least of which how people use that feature, and could make it easier to use over time.
Well, at least they're trying to demonstrate awareness and explain their recommendations, even if they come up short.
Awareness? Am I missing something here because most people I've talked to (several hundred for the record) feel like Netflix recommendations are hit or miss, a black box that plateaus after a while. If anyone can explain to me how Netflix conveys their awareness I'd love to hear it.
Speaking of explanations, sure, showing that I'm being recommended a movie because of some other movies is sometimes helpful, but the rest of the time it just reveals how poorly they understand me. It's can be pretty obvious that the algorithms have no clue why I actually like certain movies, after all, how could they? There are tons of things to like/dislike about any given movie, but when that gets boiled down to 5-stars all that context gets lost. Diminishing returns and noise will keep them from really understanding my tastes with their current system.
Why am I ranting? Because I want to be understood but Netflix and other personalization services still feel so damn impersonal. It's frustrating.
Anyways, we're working on a solution, sign up for our alpha at http://tagbax.com or comment/vote to tell me how right/wrong I am. Thanks for reading.
For example, I should probably be planning for more future dimensions (ratings, metadata, etc.) so I can manipulate their interactions in a much more elegant way as we add new data, rather than trying to manually flatten everything out (combining metadata and ratings into a score for each item, then calculating compatibility) for every possible combination, which seems like a much less computationally expensive approach. I probably just need to always keep every scrap of data we get, then find the right ways to combine the data into a score with one routine, and compare the scores to calculate compatibility with another. The hard part is figuring out how the score should be calculated (I'll get to that in the morning).
Seems like this rabbit hole goes pretty deep...
More importantly I just don't think you can predict what I want to watch based on factors such as history, ratings, watch times etc. External factors such as mood, recommendation from friend (not on Netflix) and curiosity almost always play a large factor in choose the thing I watch next. These things are not known to Netflix.
As a semi-educated guess I would say that simply providing a listing of popular movies based on ratings/watch times per genre as well as similarity would be enough for most people. It is easier to understand and how many people want to watch something just because it is popular anyway?
(I wonder to what extent it also constrains recommendations based on user similarity.)
http://en.wikipedia.org/wiki/Video_Privacy_Protection_Act
The Video Privacy Protection Act (VPPA) was a bill passed by the United States Congress in 1988. Congress passed the VPPA after Robert Bork's video rental history was published during his Supreme Court nomination. It makes any "video tape service provider" that discloses rental information outside the ordinary course of business liable for up to $2500 in actual damages.
(Covers DVD rentals too, and apparently paid-for streaming content.)
Wow, never heard of that law before...
http://www.theamericanporch.com/bork4.htm
--------------------------------------------------------
When I arrived that day I told the assistant manager I was thinking of writing about Judge Bork’s video tastes.
“Cool,” the assistant manager said. “I’ll look.”
While I stewed in a sudden outbreak of conscience – what if Robert Bork only rented homosexual porn…or slasher flicks…or (the…horror…) Disney? – she went upstairs to eyeball the records, returning a little glum.
“There sure are a lot of them,” she said. “Is it okay if I make a Xerox copy?”
Next day, while visiting Jack Shafer, I asked if an article on Judge Bork’s video rentals would interest him.
“You can’t do that, “ he said. “That’s illegal.”
“No, it isn’t,” I said. “Judge Bork said so.”
Okay, Shafer said. Write the story.
--------------------------------------------------------
Crazy!
Compare to the EU laws:
http://www.machinelearning.org/proceedings/icml2007/papers/4...
Also the instant que is not in the order we added the items in, and seems to be arbitrary. It's just a terrible service with stale content. We'll be cancelling soon.
HOWEVER, I do agree with you. I habitually do not use the web UI and only use Netflix through my two Rokus. The web UI will tell you its on DVD, my Roku won't.
Problem is, my Roku used to, so clearly the functionality exists, they just shut it off. I'd like it to be put back, honestly.