Hacker News Reader

Hi p1esk, RNNs can be more tolerant to noise because they can learn transient or dynamic attractors. If the inputs move an RNN into an attractor, small changes due to noise make little difference to the state.

Recurrence can help with robustness in some other very important ways as well.

Citations for this dates from the 80s and 90s. I don't know the best reference offhand. You could look at some old Hinton stuff if you're a fan. Lots published on this.

View on HN

That’s handwaving. The only way to find out if rnns are more robust to noise than cnns is to test them on the same task, with the same inputs and the same noise. Preferably using similar number of parameters and achieving similar accuracy before noise is applied. Then gradually increase the amount of noise, and compare the impact.

Nothing like this has been published AFAIK.

After you have the results of this experiment you can try to explain them with attractors and what not, but I would be surprised if there was much difference. Would make a good paper though!

Reply on news.ycombinator.com