Hacker News Reader

Comment by pakl | Hacker News Reader

Feedforward CNNs cannot tolerate as much noise and real-world variability as RNNs.

Citation?

There is this new type of trolling where someone just puts in bare minimal counterarguments and asks for citations knowing full well it's labour intensive to do so. If your not trolling.. your not doing a good job of avoiding these comparisons. The onus should be on you here to provide citations first that dissprove the person your replying to.

I'm not trolling. The statement that RNN are somehow more noise-tolerant than CNNs does not make sense to me, and is not based on any literature about noise tolerance in NNs that I'm familiar with. Also, no arguments have been provided as to why this could possibly be the case.

Hi p1esk, RNNs can be more tolerant to noise because they can learn transient or dynamic attractors. If the inputs move an RNN into an attractor, small changes due to noise make little difference to the state.

Recurrence can help with robustness in some other very important ways as well.

Citations for this dates from the 80s and 90s. I don't know the best reference offhand. You could look at some old Hinton stuff if you're a fan. Lots published on this.

That’s handwaving. The only way to find out if rnns are more robust to noise than cnns is to test them on the same task, with the same inputs and the same noise. Preferably using similar number of parameters and achieving similar accuracy before noise is applied. Then gradually increase the amount of noise, and compare the impact.

Nothing like this has been published AFAIK.

After you have the results of this experiment you can try to explain them with attractors and what not, but I would be surprised if there was much difference. Would make a good paper though!

> The onus should be on you here to provide citations first that dissprove the person your replying to.

Unless you're dealing with opinions, I disagree. The onus is on the person trying to give evidence to actually give evidence.

There was a post here 9 days ago discussing "intellectual DoS attacks" which basically work like this: https://news.ycombinator.com/item?id=19293036

Closely related to sea-lioning, in my opinion.

Absolutely.

Reply on news.ycombinator.com