Comment by NumberSix | Hacker News Reader

Comment by NumberSix | Hacker News Reader

The article is misleading if not false. Neural nets were hot in academic AI research 30 years ago (1988). The original Perceptron had fallen out of favor in part because of arguments that it could not implement a exclusive or (XOR) in Minsky and Papert's book Perceptrons

https://en.wikipedia.org/wiki/Perceptrons_(book)

Neural nets fell out of favor in the 1970's but came back and became hot in the early 1980's with work by John Hopfield and others that addressed the objections.

https://en.wikipedia.org/wiki/John_Hopfield

Practical and commercial successes were limited in the 1980's and 1990's which led to a reasonable decline in interest in the method. There were some commercial successes such as HNC Software which used neural nets for credit scoring and was acquired by Fair Isaac Corporation (FICO).

https://en.wikipedia.org/wiki/Robert_Hecht-Nielsen

I turned down a job offer from HNC in late 1992 and neural nets were still clearly hot at that time.

Some people continued to use neural nets with some limited success in the late 1990's and 2000s. I saw some successes using neural nets to locate faces in images, for example. Mostly they failed.

AI research is very faddish with periods of extreme optimism about a technique followed by disillusionment. One may wonder how much of the current Machine Learning/Deep Learning hype will prove exaggerated.

Also, traditional Hidden Markov Model (HMM) speech recognition is not rule based at all. It uses a maximum likelihood based extremely complex statistical model of speech.

Hinton himself did just fine in terms of academic popularity in the '80s and '90s. We can look at his citations on Semantic Scholar:

https://www.semanticscholar.org/author/Geoffrey-E-Hinton/169...

Citations to his papers have been rising steadily from between 88 and 107 in 1987 to between 685 and 826 in 1999. That's hardly an unpopular researcher.

And for a bit of comparison with other machine learning researchers, here's a link to a data set of family relations from a 1986 paper by Hinton:

https://archive.ics.uci.edu/ml/datasets/Kinship

At the bottom of that page, in the Relevant Papers sections there's two links to two papers using the data set, one Hinton's own paper that introduces it and one by Quinlan.

Clicking on the [Web Link] links for the two papers, I can see the references to those papers. There is a single reference to Quinlan's paper. There are 43 to Hinton's, of which all but 6 are from 1999 and earlier. And those are not self-references, neither references by Bengio, Le Cun et al. If there is a clique, it is hard to see it.

So there was a lot of interest to Hinton's work even in the years he was supposed to be "exiled to the academic hinterland" as another article said.

> AI research is very faddish with periods of extreme optimism about a technique followed by disillusionment.

Truth! I happened to go through grad school at a time when SVMs and kernel methods were cool, neural nets were the opposite, and Hinton and his students were oddball outcasts at NIPS. Kudos to Hinton for sticking to his story until the current "deep learning" hype wave presumably allowed him to buy a yacht and an island. I imagine we'll be hearing about some other non-linear optimization technique in a few years.

The XOR story is misleading. :-) The Wikipedia page does a good job in what the book actually proved:

"What the book does prove is that in three-layered feed-forward perceptrons (with a so-called "hidden" or "intermediary" layer), it is not possible to compute some predicates unless at least one of the neurons in the first layer of neurons (the "intermediary" layer) is connected with a non-null weight to each and every input. This was contrary to a hope held by some researchers in relying mostly on networks with a few layers of "local" neurons, each one connected only to a small number of inputs. A feed-forward machine with "local" neurons is much easier to build and use than a larger, fully connected neural network, so researchers at the time concentrated on these instead of on more complicated models."

Reply on news.ycombinator.com