If I recall, BellKor made many, many models based on Gradient Boosted Decision Trees, Restricted Boltzmann Machines, and kNN. They tried many different feature subsets, added temporal weighting, and tried many reduced-dimensionality representations (SVD, NMF). They then stacked them all together into one final ensemble whose RSME beat everyone else's on a hidden validation set.

In a production environment, this is probably an insane amount of transformation, feature extraction, and classification for marginally little gains in precision (as defined here). But I'm only a year or two in to building production-environment classifiers, and nothing at Netflix's scale (though not tiny either-- it is a problem if I can't do feature extraction and high-precision/-recall* classification within a few milliseconds).

* - mid 90s, for a hard NLP/social graph problem.

I am a 'Kaggler', I have done well (prize winner) in some past competitions and I could not agree more.

In the last week of a competition, you go into kitchen sink mode trying anything and everything to squeeze the last few points out of your models. The objective of the competition from the competitors standpoint is to win, not create a solution that could be deployed into the sponsor's production environment. On going maintenance of the model is not a consideration.

As far as the netflix competition goes though, the final solutions did help publicize the potential of ensembling and RBMs - I am grateful for that. My personal approach to the competitions is to use them as an opportunity to try out new modeling techniques (i.e. RMBs, deep beliefs etc...) on real world data. It has the added bonus of potentially paying off with prize money (as long as you don't get screwed[1]).

I would like to hear from past Kaggle sponsors and see what they have done with the winning models.

[1] http://www.gequest.com/c/flight/forums/t/4284/acknowledging-...

Largely what textminer said. In fact they didn't use anything from the grand-prize winner because it would require too much engineering effort and because their concentration on the problem shifted. They did however use the two best methods from the "progress prize", so I wouldn't call the contest a failure.

>If you followed the Prize competition, you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later. This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line. We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment. Also, our focus on improving Netflix personalization had shifted to the next level by then.

From http://techblog.netflix.com/2012/04/netflix-recommendations-...