Hacker News Reader

> the technique is fairly new and used in deep learning, popularized by some of the guys that popularized deep learning, usually mentioned in conversations about deep learning

Logistic regression with regularization is fairly new? 'Pioneered' by the same people as deep convolutional neural networks? Are you certain about this?

View on HN

I think you need to define regularization. L1/L2? Yes those are old.

Drop out itself IS fairly new[1]. As well as its newer cousin drop connect.

I agree neural nets themselves are basically just a crazier parametric model. Many of the things we do to modify the gradient are applicable to logistic and other simpler regression techniques as well.

[2]: https://cs.nyu.edu/~wanli/dropc/dropc.pdfhttp://www.cs.toron...

Reply on news.ycombinator.com