Hacker News Reader

Comment by argonaut | Hacker News Reader

This is neither basic nor an introduction, and has several inaccuracies. My guess is the info is from the 90s era of neural networks.

Reply to below: it uses pretty complicated language and doesn't explain many terms (e.g. activation functions, supervised learning, etc. are not explained). Errors: the statements the article makes about the error surface are really misleading and in some places wrong. Same for the statement about sigmoids. Some of their definitions are also wrong today, such as "epoch." The list goes on.

View on HN

What do you think is wrong or outdated with the article? It's not that far off how ANN are described in the beginning of current scientific articles. Backpropagation is definitely still relevant even though changes can be made to it and the networks have grown in size and changed in layout. The basics are more or less the same and from my quick scan they seem accurately described here. What is it you expect from a simple intro?

While I don't share his total disdain for the article, he got the date spot on, and I'd say that

- no mention of non-sigmoid activation functions - calling overfitting 'grandmothering' and - diving comparatively deep into gradient descent (which while absolutely the most common, is not the only way to train a neural network), while being an inch deep on practically every other aspect.

Its a bit of a confused intro that simultaneously tries to teach you the basics while throwing jargon at you ('delta-rule', 'beta-coefficient', 'hyperparaboloid', etc.).

What actually is new since the 90s, besides convnets?

Is ML becoming more prevalent because of theoretical breakthroughs, or is it because of hardware improvements, or perhaps because there is more training data available now?

What enabled the ML revolution we're witnessing now is mostly the advent of powerful and low-cost GPGPUs and the abundance of training data, as you mentioned. Of course, there are theoretical breakthroughs and interesting new applications [1], but it wasn't the deciding factor. Even ConvNets aren't new. For example, AlexNet is basically a much larger reimplementation of the original Lenet idea [2] from 1998.

This is not to discredit any new work that's being done. I think it's awesome to see so much progress in a field that's certainly on its way of defining an era. I'm just pointing out that we have known about most of the fundamentals for a long time.

[1]: https://tryolabs.com/blog/2016/12/06/major-advancements-deep...

[2]: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf

It's not about newness. There are several things that are flat out wrong in the article.

Reply on news.ycombinator.com