Hacker News Reader

Comment by vonnik | Hacker News Reader

Recursive neural networks have not seen wide adoption in the deep learning community. To my limited understanding, this is because their performance depends on the quality of the parser you use, which makes them fragile. Has anyone had experience using them?

View on HN

Parsers are pretty good and getting better, have you seen: https://research.googleblog.com/2016/05/announcing-syntaxnet...

This is a very successful application of recursive NNs in Question/Answering domain: https://cs.umd.edu/~miyyer/qblearn/

Not really my area, but RNN variants such as LSTMs/GRUs were (are?) the state of the art in NLP tasks.

Recursive Neural Networks are not the same as Recurrent Neural Networks (RNNs).

The repo uses RNN for Recursive Neural Network and specifically calls attention to its LSTM implementation.

Thanks for pointing this out. The same acronym is used to refer to both versions. I like the description of Recursive Neural Nets versus Recurrent Neural Nets in this Stack Overflow post [1].

> A recursive network is just a generalization of a recurrent network. In a recurrent network the weights are shared (and dimensionality remains constant) along the length of the sequence because how would you deal with position-dependent weights when you encounter a sequence at test-time of different length to any you saw at train-time. In a recursive network the weights are shared (and dimensionality remains constant) at every node for the same reason.

In the TreeRNN (Tree Recursive Neural Nets) Github project and associated paper [2], there is indeed an implementation of a TreeLSTM which is a recursive LSTM inspired by the recurrent version.

[1]: http://stats.stackexchange.com/a/158995/92040

[2]: http://www.aclweb.org/anthology/P15-1150

Reply on news.ycombinator.com