Hacker News Reader

Addressing your second question. Informally, dropping nodes fights overfitting by creating subsample architectures of which are essentially thinned out networks of the one you've designed. Having trained on these sub nets means you've effectively combined the learning of a few different models and in doing so have generalized beyond the capabilities of your original "single" architecture.

My understanding is that it avoids overfitting when data points are highly correlated.

For example, if you use image augmentation to generate additional data, your augmented images are going to be highly correlated to their parent image leading to overfitting of the data. By using random dropout, this overfitting can be somewhat mitigated.

For linear models it can be shown to be equivalent to weight decay. For nonlinear ones, it empirically behaves as a regularizer.,