Hacker News Reader

Comment by henryl | Hacker News Reader

I find Netflix's recommendations to be far superior. Participants in the Netflix Prize have found that content-based filtering (such as "nano categories") actually hurts relevancy (in RMSE terms) when working with large data sets. That is, when you have a copious amount of data, it is usually better to let the data speak for itself. Similarly, it has been noted by Google and others that the better your data, the less impressive your algorithms need to be. Most people think Google's success is mainly attributed to magic algorithms, but most of their effort of late has been in increasing the quality and breadth of their data.

http://anand.typepad.com/datawocky/2008/03/more-data-usual.h...

View on HN

I was under the impression that Netflix looked at your ratings, found people "similar" to you, and made recommendations based on that. Since signing up for Netflix, I"ve found that at least on the UI side, it's as you describe: driven by actor, genre, etc.

The beauty about the raw data approach is that it finds people with similar preferences to you. This is, I think, the real manifestation of O'Reilly's "Web 2.0." Rather than a semantic web, with ontologies and categorization the data do, indeed, speak for themselves.

Even if we were able to extract categorizations based on the preferences in the underlying Netflix data, it'd be difficult to map them to actual categories that we're familiar with. I'd envision it more like a PCA decomposition, where the principle components will be the strongest characteristics among each clique of like-minded movie watchers.

But alas, my Netflix home page is filled with crappy movies (the Matrix has a near-5 rating, but most other movies with its actors are complete crap). Instead I rely on friends with similar preferences for recommendations... which is what Netflix was supposed to offer. If that functionality is hidden somewhere, then they need to do a better job of exposing it.

Content-based data may not help with Netflix prize, but they do help to quickly drill down to what people want to see. You can pick a "nano category" based on the meaning of the words in it, rating data can be superior but cannot be used interactively in such a manner.

I've thought of doing a service like this movie search engine, maybe I still will. Search engines don't really leverage interactivity as much as they should. Consider a simple example: I choose Pi as a movie I like and Drama genre and get a search box to search within dramas similar to Pi. Providing context can go a long way towards getting what the user wants.

Reply on news.ycombinator.com