Hacker News Reader

Comment by paulsutter | Hacker News Reader

The original Deepmind paper [1] is based on a really smart idea. Algorithmic development relies on measuring the performance of any proposed algorithm. For reading comprehension, performance is evaluated using Q&A about the corpus. Its difficult to find a large corpus with a comprehensive set of questions about the content.

Deepmind is cleverly converting the Daily Mail article summaries into questions by removing a proper noun. For example:

Question: Producer X will not press charges against Jeremy Clarkson, his lawyer says.

Answer: Oisin Tymon

They are using the Daily Mail corpus to develop their algorithm, and that's smart. They aren't relying on it as an important source of information. Maybe all you guys with the dismissive comments have a better idea?

[1] http://arxiv.org/pdf/1506.03340.pdf

EDIT: Thanks Otik, reworded the opening sentance

View on HN

For a moment I thought you were calling the Mail "a great paper", but actually the simple language used by it is probably quite good for this purpose.

Of course, it does mean that if you ask Deep Mind about the cause of anything negative it will probably tell you "immigrants".

or "benefits scroungers"

edgy stuff

I agree with you - it's a neat idea, though I haven't read the paper yet.

I wonder if it has explicit knowledge that they'll always remove a proper noun. Eventually the daily mail will write some article about a place causing or curing cancer, and the poor thing will break down trying to find X in X causes cancer from that corpus :)

On another note, I wonder if they could train it on Buzzfeed-esque content without changing the headlines. "You won't believe who Jeremy Clarkson punched!" -> the answer

The daily mail should not be a source of information at all, though. Any greater weighting than zero is too much. If it's only used as a test for the algorithm then I guess it can be OK, assuming the Mail's broken logic doesn't get in the way. It's the step above dream logic, really. Deepmind might have problems abstracting too much over the text because of this. Trying to combine information from several different articles would be basically impossible.

Anyway, how about using Wikipedia instead? I can only guess they're using the Mail because they think it's funny.

"I can only guess they're using the Mail because they think it's funny."

Ironically, given your self-professed superior tastes in reading material, Deepmind seems to have a better ability to read, comprehend, and answer questions about articles than you do.

So they're using a corpus as a determining factor for quality assessments, but you don't think that makes it an important source of information?

Thanks for the summary. I really didn't understand what the article was trying to say they where doing.

Reply on news.ycombinator.com