Theano Implementation of Tree Recursive Neural Networks | Hacker News Reader

Theano Implementation of Tree Recursive Neural Networks

github.com

53 pointssndean8 comments

Recursive neural networks have not seen wide adoption in the deep learning community. To my limited understanding, this is because their performance depends on the quality of the parser you use, which makes them fragile. Has anyone had experience using them?

Not really my area, but RNN variants such as LSTMs/GRUs were (are?) the state of the art in NLP tasks.

Recursive Neural Networks are not the same as Recurrent Neural Networks (RNNs).

The repo uses RNN for Recursive Neural Network and specifically calls attention to its LSTM implementation.

Thanks for pointing this out. The same acronym is used to refer to both versions. I like the description of Recursive Neural Nets versus Recurrent Neural Nets in this Stack Overflow post [1].

> A recursive network is just a generalization of a recurrent network. In a recurrent network the weights are shared (and dimensionality remains constant) along the length of the sequence because how would you deal with position-dependent weights when you encounter a sequence at test-time of different length to any you saw at train-time. In a recursive network the weights are shared (and dimensionality remains constant) at every node for the same reason.

In the TreeRNN (Tree Recursive Neural Nets) Github project and associated paper [2], there is indeed an implementation of a TreeLSTM which is a recursive LSTM inspired by the recurrent version.

[1]: http://stats.stackexchange.com/a/158995/92040

[2]: http://www.aclweb.org/anthology/P15-1150

Parsers are pretty good and getting better, have you seen: https://research.googleblog.com/2016/05/announcing-syntaxnet...

This is a very successful application of recursive NNs in Question/Answering domain: https://cs.umd.edu/~miyyer/qblearn/