The title makes it sound like the prize ended up being pointless. The article says otherwise.
The title makes it sound like the prize ended up being pointless. The article says otherwise.
I work on the cinematch team at netflix. The several hundred blended algorithms that was output of the grand prize winning team is very... impractical... for netflix' needs. We cherry picked (and then modified) the best of the bunch.
The article goes on to say "...you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later...We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment."
The title is completely accurate.
Accurate, but a half truth. The prize was for a 10% improvement, but before that solution was produced they had already improved by 8.4%. The headline makes it sound like the improvement from zero to 10% was not worth the engineering cost, but really it was the improvement from 8.4% to 10% which cost too much.
It was pure luck that the threshold for the million-dollar prize was crossed. If it had been arbitrarily set at 11% (as opposed to 10%), then there's a good chance the million dollar prize would have never been paid out.
The advantage of paying out K top prizes is that other teams that don't win outright may have developed additional useful models or insights (or used more computationally efficient algorithms), and you may access to these in this manner.