What do you guys think the next big thing after deep learning is and when will it come?
Deep neural networks are great at recognizing patterns, but they still don't give us much in the way of reasoning. So, "I should start cleaning now because lunch is at 1:00 and it takes uncle Bob 30 minutes to drive over here..." kind of stuff is - so far - pretty much outside of the realm of ANN's.
If / when we have systems that can utilize the best of probabilistic pattern matching and formal logic, I think we'll see a dramatic improvement in the "intelligence" available.
Note that there is no reason, in principle (at least that I'm aware of) that "formal logic" might not ultimately turn out to be an emergent aspect of a sufficiently complex ANN. Given that our brain can do formal logic, and does at times, and is "just" neural networks as far as we know, it seems likely. That said, in the short-term, I'm guessing we'll have hybrid systems that explicitly feature various AI techniques that have been developed over the years, sharing some representations (of concepts, knowledge, etc.). Maybe the "thought vectors" that Geoffrey Hinton has spoken of could be a step down that path. I'm also curious to see if something like rule induction using CN2 might not play a role. Or maybe it'll involve "semantic networks", something like ConceptNet. Who knows?
If I had to summarize, I'd say the answer is something like "coming up with some form of knowledge representation that allows us to share 'thoughts' or 'knowledge' between disparate AI/ML systems".
In principle, one reason against the ANN purist's approach for replicating human intelligence is that it ignores all the things our brains already have programmed from the moment we step into the world; we obviously aren't anything close to a blank slate. So I think there's a burden on advocates of pure ANN to explain why our core intelligence or "formal logic" is in fact independent of this pre-programming.
That's a fair question, and I take no position on it. My take on ANN's is that they are useful tools. I'm not particularly interested in the degree to which they accurately replicate how the human mind works. I assume, perhaps too quickly, that the analogy is pretty close, but I admit that I can't prove that.
In either case, I still feel relatively confident that formal logic could emerge on top of a sufficiently complex ANN. One thing I don't know off hand, which would be interesting, is whether or not any specific research has bee done on proving or disproving that assertion.
It's closer to a gut feeling than anything. I believe if fairly strongly, but I haven't done any research in terms of trying to prove the point one way or the other. Don't read too much into it... it's just something I think. I may well be wrong.
I believe that this statement dismisses the fact that humans have plenty of time to train their brains before the birth based on multiple sensory inputs.
I get the impression from the 101 child development textbooks that infants display an impressive number of schemas and knowledge despite little, if none environmental exposure, but I'm not familiar with arguments linking these schemas and knowledge to exposure in the fetal stage.
Babies are exposed to the environment to some degree (noise, gravity etc) and it would be very dismissive to ignore it.
One thing that is for sure though is that once we have artificial general intelligence, everything changes infinitely forever.
"Hopefully machine learning techniques give us some sort of building blocks for a general intelligence."
"Hopefully knowledge graphs give us some sort of building blocks for a general intelligence."
"Hopefully Bayesian models give us some sort of building blocks for a general intelligence."
"Hopefully rule-based systems give us some sort of building blocks for a general intelligence."
"Hopefully LISP gives us some sort of building blocks for a general intelligence."
"Hopefully computers give us some sort of building blocks for a general intelligence."
People have said approximations to all of these over time. All of these probably are building blocks. But there's no reason to believe that we're about to build the top of the tower.
I believe we should understand AI in terms of what it can do for us now, and that AGI keeps appearing to be 50 years away because it's actually centuries away and we don't even understand what we don't understand yet.
I really don't see why DNN change the game fundamentally. I mean graphical models, they really changed the way of thinking in the field. DNNs in contrast are really still fundamentally building on things built by Yann LeCunn and gang.
How do you think it will change everything infinitely forever? Are you confident this would be a positive change for us and all species on earth? This is where I'm a little lost about the enthusiasm for AGI but I'm curious what others think?
The first few improvements may be tough for it, but when you consider that it could think 1000s of times faster than humans, you can imagine how this could happen very quickly.
And then once you assume that it can recursively improve itself, we enter the realm of "the intelligence with the IQ of 5000."
The person with the highest IQ ever was 190 [1]. Imagine the kind of things we could do with an intelligence of 5000. We're talking atomic reassemblers, grand unified theories, and (my dream) a personal paradise Matrix for everyone [2].
[1] https://en.wikipedia.org/wiki/Marilyn_vos_Savant [2] http://www.smbc-comics.com/?id=3183
Note, this sidesteps all ethical alignment issues. This is assuming that the intelligence is on the same team as us. My thought process is "we're all going to die anyway so let's give it a shot," but a lot of people disagree with me (which is fair).
I think AI will split into two sets, biological inspired that will be general purpose. The other will be tool, much moreasier alien and specific to a domain. Bit like how aeroplanes don't look like birds.