Hacker News Reader

Comment by aswanson | Hacker News Reader

It may be simply mathematically impossible to get to 10 percent given the observable variables. Maybe netflix knows this and just gets synthesized research on the cheap.

View on HN

I'd guess the machine learning guys by this point understand the data set at least as well as Netflix does.

The machine learning guys have only a subset of the full dataset to work with. They have to send their algorithms to Netflix for testing on the full dataset. So it may be possible Netflix knows something they don't. It does seem quite strange how close and yet how far that 10% mark has been.

Because they are testing it vs a fixed data set teams can gain information about that data set based on their submissions. Which is why several teams mix and mach several different approaches. They can increase the weight of each approach over time and game the system.

Edit: I am not saying they are intentionally cheating rather it's one of the reasons that approach works.

Hmm. What could Netflix know, though? Is it even possible to quantify how hard it is to improve a recommendation algorithm by a certain amount?

Maybe there is an inherent noise floor for the optimal estimator for the dataset they have found or bounded.

Wouldn't argue with that, but they may have run up against a bound, multiplied it by a constant, and said to themselves "highly unlikely that anyone can beat this, so lets put it out there and see if someone out there can beat it."

So far no one has and they have gotten a ton of research on the cheap, like the current team's for 50 k USD. Smart.

They are getting it on the cheap. They've got many teams of top-notch folks working on building an algorithm tailored for their data-set and problem definition. For what they're getting out of this, $1m is a stupidly low price. If they've incorporated the stuff that's come out so far, they've already boosted their sales by several million based on the results of the contest.

If the goal had been 5%, you would say: - It may be simply mathematically impossible to get to 5 percent...

And they are already at 9.56%.

No I wouldn't have, because I could only make a statement like that after seeing performance information. Since at least 40 teams beat that performance the first year, I would have said they set the bar too low.

And why do you think 10% is impossible but 9.5% is not?

I said they may have found a bound. I didn't say it was; it said there may be a limit. Re-read the ancestor.

You are playing with words.

The point is that there's really not much difference between 10% and 9.56%.

Netflix's Grand Prize could very well had been 9.64% and it would already had been won, despite everybody saying one year ago that 9.64% might be impossible given the dataset.

We will talk again in one year to see if 10% may be impossible.

Barring either of us getting hit by a beer truck or moving on to a new site,or the apocalypse you're on. Catch you 12/11/09!