Hacker News Reader

Comment by jefft255 | Hacker News Reader

I don’t like how they oppose deep learning with « logic based AI ». When Hinton got great results, he wasn’t competing against logic based AI but rather other machine learning approaches like SVMs.

View on HN

I dont think he was competing. ANNs and boltzmann machines in the 80s seemed was an exploratory field not really part of statistical ML. At the time people were indeed competing with logic-based AI. SVMs came later and even then i believe, NNs had a different history and were not motivated from competition to SOTA techniques like SVMs. When the success of trainig came after the mid-2000s that's when ANNs started being called "ML". At least that's my memory

Indeed, he wasn't competing. In fact, he wasn't moving; he kept doing what he had always been doing until, luckily, something, that he had nothing to do with, changed and allowed NN methods to work better. So I view him as essentially standing still, frozen in time, like a clock stuck at 6 o'clock. And, just like that clock, he would inevitably be correct (at least twice a day). He had nothing to do with the tools (higher memory & CPU speeds) that made his methods work, he just kept doing the same thing over and over until, one day, by accident, something important changed: his lab bought newer, faster computers.

He's not exactly Louis Pasteur is what I'm saying!

he s still the coinventor of boltzmann machines, and one who kept the field alive.

We call this "skating to where the puck is going to be".

I call it "doing the same thing over and over again, hoping someday you get lucky." I think there's a commonly-used expression for that behavior.

Hinton got lucky: something he had nothing to do with changed, making him look like a stoic hero.

A winning strategy or just a lazy path?

Well, Hinton himself makes a big deal of this in his Turing award speech. See: https://fcrc.acm.org/turing-lecture-at-fcrc-2019

Me neither. In fact, both will be important components of future AI. I like to see deep learning as intuition, and logic-based AI as formal reasoning.

Intuition allows systems to avoid search problems, to some extent. Formal reasoning introduces structure.

Sure, but for someone who does not know what an SVM is, the big-picture takeaway is that a particular version of "soft" statistical learning (deep learning) had succeeded where "hard logic" approach failed.

Just asking: hard logic failed in what area?

Computer Vision would the the most likely example. It’s quite hard to come up with “hard rules” for distinguishing cats and dogs.

Carrots are vegetables. Vegetables are edible. Is this a picture of something edible? Having a bunch of propositions in a database is not much help if you can't relate them to messy sensory input. It turns out that making sense out of mess was a hard problem. Eventually, logic* will have to be brought in again, but at a higher level, to help out.

* Ofc Hinton's approach is based on logic too, but in the form of probability, calculus etc.

but with probe metrics and other experiments we are retroactively explaining why the big breaktroughs actually work, and these can inspire and fix the big breakthrough by putting them on more rigorous foundations. At the end you get a formal explanation of the conditional probability that given a carrot with 6DoF and such and such parameters (age, size, ...) in such and such lighting conditions results in that probability distribution for pixel values.

Speech recognition is one such case. We thought that by breaking down the incoming speech into phonemes we would be able to get words from the bottom up. That never worked too well, whereas statistical speech recognition has been much more effective.

Reply on news.ycombinator.com