Beyond Amazon: How to Make Recommendations Smarter

"Researchers won’t be able to look at the data, but they will be able to dump their algorithms in and have the box spit out results, which the researchers can then use to refine their hypotheses."

I may be thick, but I'm not sure how much value there's going to be if researchers can't get access to the data. Anonymized data would still be more useful than a black box. I can think of some general things that can be tested, but it still seems like this will be an uphill battle for people wanting to use it.

But... I'm not a maths/algorithm guy, so perhaps this will still be useful to people.

What a simple idea that will go a long way. There are great many algorithms/models out there that are seemingly great and do wonders on toy data, but in the real world fail not only because the data isn't comprehensive (it isn't sampled well), but also because people misuse existing models and tweak them enough to work for that data and those tweaks make it on acclaimed journals/conferences. These kinds of efforts will help tremendously to filter out all the noise because now, you'll have to show results for real-world data to argue that your algorithms can generalize and/or that your tweaks actually matter.

Besides Amazon, Google has taught us that more data is better and sometimes less complex algorithms that scale but give you only a marginal hit on accuracy are preferred; ie, cost functions matter, and that's something you don't get practice with in the academic world.

When you think about data sets to boost performance, think outside the directly dependent ones. There is so much data out there now and it's not being used. This I think will change soon.