The Unreasonable Effectiveness of Recurrent Neural Networks (2015) | Hacker News Reader

The Unreasonable Effectiveness of Recurrent Neural Networks (2015)

karpathy.github.io

65 pointssidcool6 comments

Good article, but ugh, please leave Shakespeare alone if (as you freely admit) you don't know anything about Shakespeare. Just because the real thing sounds strange to you doesn't mean that your "samples" are anything but nonsense.

edit To (preemptively) stress that I am not trolling: I find the topic and the writeup interesting and useful, and I appreciate the work that people do to share their experiences. But this is not a good example of the RNN working; it's a good example of the RNN not working, and should either be omitted or presented at least neutrally, instead of glibly saying

> I can barely recognize these samples from actual Shakespeare :)

It works about as well/poorly as the other examples (Paul Graham, Wikipedia, math in LaTex). The samples from all of the examples are nonsense. What's interesting is that they, mostly, follow the form of the original.

Fair enough. My objection was that the OP implies this is a passable substitute, but okay, in the larger context it's clearly a joke.

I think it's a usual tendency of NNs to produce output that looks meaningful to non-experts, yet is complete gibberish to experts. At a PLDI2016 keynote, the presenter gave this example from running an RNN on all PLDI submissions:

"We present a convenient alias analysis that attains a number of queries in a typedef style." (fuller context, with the surrounding sentences being more bizarre may be found here: http://www.cs.ucsb.edu/~ckrintz/PLDI2016-keynote-final-zorn....).

Previous discussion: https://news.ycombinator.com/item?id=9584325 (458 days ago, 211 comments)

Really interesting article (missed the previous post)!

How far is this from being implementable for a practical application such as e.g. a "spellchecker and improvement ideas" for coding? E.g. train with "good" code from open source projects, have it running and highlighting areas with potential errors/improvement potential while you code (not counting trivial errors caught by e.g. lint tools).

Reply on news.ycombinator.com