Hacker News Reader

There've been cycles & fads in the meantime, though.

I remember that in the early 90s, neural nets were supposed to be the huge new thing. Many scanners shipped with built-in OCR, the Apple Newton had handwriting recognition on a PDA, and my Centris 660AV could do speech recognition & text-to-speech out of the box.

But they ultimately weren't powerful enough to satisfy customers or meaningfully change how people interacted with computers, so they failed in the market, and the hype cycle moved on to the World Wide Web.

I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits.

> I guess this shows the power of continuing to study something when the fad goes away, so that you're well positioned to capitalize when the next fad hits.

So true. VR popped up in the 90/00s (Virtual Boy, VRML, etc.), went nowhere, and here we are again.

I suspect something similar will happen with blockchain/cryptocurrency. Only once all the hype and speculation dies off will meaningful uses for the tech become evident.

Are you trying to suggest that Cryptokitties aren’t meaningful? Blasphemy :-)

The hard part is surviving the 20 year drought of research funding in the meantime. Most researchers want to hang onto an idea and develop it until it changes the world but they need to keep their job, so they follow the funding and that means following the fads.

So true. I did part of my PhD on reinforcement learning, finished in 2004. I couldn't get money for continuing work on the topic (wasn't cool), so ended up getting into P2P and Big Data. Still do a bit of ML, but it's not core anymore.

Could not have said it better. Hinton is one of the most important contributors to the field, but giving him all the credit is a disservice to others who made important contributions. Probabilistic modeling was popular way before DNNs got big. At that point it was already clear that logic based inference was only so powerful, and different methods were necessary.

>> At that point it was already clear that logic based inference was only so powerful, and different methods were necessary.

Er, logic inference is "only so powerful"? The first order predicate calculus (first order logic) is Turing-complete and there is plenty of maths that prove the soundess and completeness of various logic inference rules. In other words, if you can compute a function, you can compute it with first order logic.

The switch from symbolic to statistical AI happened partly because of the AI winter that cut the funding to all AI, which at the time was primarily symbolic AI, partly because it became evident that developing and maintaining huge databases of logic rules was inefficient. Some of the early work in machine learning focused on overcoming this inefficiency by inducing rules from data.

>> The first order predicate calculus (first order logic) is Turing-complete

Brainfuck is turing complete, does that make it as powerful as a general purpose programming language such as Java?

I'm not criticizing first order logic, it has use cases no doubt. On the other hand there are many cases where codifying knowledge into a we formed KB isn't practical. Some of these use cases are better suited to probabilistic approaches, deep learning, and some are unsolved. That pretty clearly puts limits on the power of FOL.

Also, thanks for mansplaining turing completeness.

>> Also, thanks for mansplaining turing completeness.

Snark is not OK:

Be civil. Don't say things you wouldn't say face-to-face. Don't be snarky. Comments should get more civil and substantive, not less, as a topic gets more divisive.

https://news.ycombinator.com/newsguidelines.html

>> It was well established that the future of teaching computers came from statistics and not rules in the 80s/90s by researchers like Michael Jordan (https://en.wikipedia.org/wiki/Michael_I._Jordan) and his students.

As a counterpoint, one of the most successful (classes of) machine learning algorithms are Decision Tree learners, whose models are decidedly symbolic.

There's also plenty of work on learning first-order logic theories with neural nets (see for instance the work of Artur D'Avilla Garcez).

The problem with rules is that it's hard to develop and maintain large rule-bases. However, it's perfectly possible to do machine learning for rules -and logic-based machine learning is totally a thing (full disclosure: it's the thing I'm doing a PhD on). You probably haven't heard of it though because data scientists tend to be good with statistics but bad with symbolic logic.

While I won't dispute that many in AI thought machine learning was the correct path, the majority of researchers in machine learning in the 2000s were dismissive of neural networks as the way forward. When I started my PhD in 2007 with the specific goal of studying deep learning, many of my peers told me this was a very bad career move, because neural networks were a dead technology. Convex optimization, kernels, and variational Bayesian methods were the future. They were wrong.

AI has had a lot of fads, but neural networks are here to stay, in my opinion.

Convex optimization, kernels and Bayesian methods are also here to stay. They are all in the toolbox. Optimization-based methods are still rapidly spreading in robotics, for example, even as we add CNNs to the mix.

Convex optimization is heavily used in industry for real problems. It just doesn't have the sexiness that neural networks too, so it doesn't get as much attention.

I agree. My comment about being wrong was regarding neural networks being a dead end. Convex optimization and Bayesian techniques are both incredibly useful.

I see the article just as an effort at highlighting Hinton and his AI research. I don't think he would say he had done it all. Toronto Life is saying it all.

I have just finished reading this interesting wiki titled AI winter defined as a period of reduced funding and interest in AI research:

https://en.m.wikipedia.org/wiki/AI_winter

There have been such winters since the 60's.

Didn't Hinton stick with neural networks as his primary (often only?) research focus for all that time, whereas most of those other 1000s of researchers moved in and out of neural network research as the popularity of neural networks waxed and waned?

I think that is what the article may be getting it.

In respect to the first quote from the article "computers could think like humans do -- using intuition rather than rules". Is this really a good description of neural networks? I don't think so, to me it sounds like a bad misrepresentation of what neural networks are by a writer who can't describe the way ai works without humanizing it inaccurately.

That's probably Hinton's words.

If that is the case I am eating my words

We get blasted with X greatest sports player of all time, or Y most important historical figure! None of those are really accurate, but are those people important and accomplished enough that they deserve someone lauding them so highly along with the footnote in history they receive? Whether or not they deserve the prestige they're receiving or sole credit for whatever accomplishment? I am not disputing whether or not the article's portrayal is entirely accurate.

Winner takes all, that's how human brain works... Thats why I am fundamentally pessimistic about human future. ..