The Difference Between AI, Machine Learning, and Deep Learning
blogs.nvidia.com
blogs.nvidia.com
But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states.
But just a decade ago, if you saw that a computer was able to recognize pictures better than humans, you would think there was something fishy going on, or we have achieved true AI.
My opinion is true AI is which is comparable to human intelligence, in that it is sentient and/or capable of abstract thought, not necessarily being able to hold a conversation or solve concrete mathematics problems, or dump out a story by neural networks.
I suspect it's just hearsay that's been repeated over and over again.
[1] http://karpathy.github.io/2012/10/22/state-of-computer-visio...
Again, there is a lot of coolness happening with subsets of AI research, but I don't feel that there is even a clear definition of what AI would entail - spaghetti code to get a desired result doesn't really help either, since it has to be persistent independent successes - to refer to Karpathy's article, it would have to make repeated assertions and understandings of similar photos with a high success rate to really be something spectacular.
[1] https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_i...
Bingo! The "general" part is the ability to learn new structures and task structures from environmental cues, and then construct informed prior beliefs about those new tasks and structures using causal relations to previously-observed tasks and structures. Nothing more!
Article: How Our Deep Learning Tech Taught a Car to Drive - https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3...
These neural nets are really smart, and we don't know exactly how they work, we know only in principle. But if we asked the guy who made the self driving car what is the role of the 17th neuron in the 14th layer, he would probably have no idea. Just like human brains evolve through learning, so do neural nets. Yet many people think they are just clever tricks and not truly intelligent.
I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence. For example, while AlphaGo is incredibly impressive, it can only play Go and nothing else.
I think once we have a comprehensive AI program that can play many kinds of games, can have conversations, can complete school exam questions, can write stories etc. the question of what intelligence is will become more interesting.
Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm.
https://arxiv.org/pdf/1312.5602.pdf
Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpose to maximize a reward. It can be used, and was used, in multiple domains. AlphaGo used RL as well.
Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game. In fact, if the software was set to learn more games, it could learn them in addition to Go.
Also, regarding other tasks: a neural net that recognizes cats can be easily made to recognize dogs too. A program that translates English to French can be made to translate other languages too. We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited.
Recently there has been a paper "Learning without forgetting" (http://arxiv.org/abs/1606.09282v2) that underlines this very ability to span multiple domains and adapt easily to unseen tasks and data.
> Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence.
Saying that people can do many tasks is not exactly right because a particular person can only do a few tasks, those tasks she was trained to do. If I never learned German, I don't know German. That doesn't mean the brain itself is limited. AlphaGo was only trained on Go, and its internal architecture was optimized for this one task in order to make it more efficient, but the method is general and reusable. DeepMind said so themselves, the breakthrough is not that they beat Lee Sedol, but that they used a general method that can be used to do other tasks as well. It is not a limitation of AI that we generally make systems that are good at only one thing.
If there is a limitation in AlphaGo, it is that it mastered a game where the whole situation is perfectly known (the Go board), while in reality many tasks are only partially known (such as card games, for example) so there is an extra uncertainty. But DeepMind and other researchers are working on that too.
This is exactly the kind of hubris and reasoning that brought down good old fashioned AI initially. They also had a lot of initial impressive wins with methods that looked quite general at that time.
It's important to distinguish between what sorts of games work well under this method and what sorts do not. Games that are variations of pole balancing, like Pong, fare better than more complex games like Asteroids, Frostbite or Montezuma's Revenge.
> Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game.
It's nothing of the sort. AlphaGo is a machine in the Turing sense. The neural network is a program that is the result of a search for a function specialized to playing Go. This machine, the program that the parameters across the edges in the graph represent, is logically unable to run any other program. Lee Sedol is a Universal Machine in the Turing sense, any statement contradicting this makes no mathematical sense.
> We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited.
It is well known within the literature that these models do not make best available use of information when learning. They are exceedingly inefficient in their incorporation of new information. Issues include improper adjustment of learning rates, not using side information to constrain computation, having to experience many rewards before action distributions are adjusted in the case of reinforcement learning, samples per example in supervised learning. Note that animals are able to learn without explicit labels and clear 0/1 losses.
Humans and animals generally, even in the supervised regime, are vastly more flexible in the format the supervision can take.
For an example, look into the research on how children are able to generalize from ambiguous explanations as "that is a dog" and why difficulty in learning color from this kind of "supervision" shows just what priors are being leveraged to get that kind of learning power.
See here for an excellent overview of limitations in our current approaches to AI: https://arxiv.org/pdf/1604.00289v2.pdf
> "Learning without forgetting"
That's a great paper but it does this by minimizing prediction error drift by comparing before and post performance on the old task while learning the new. I do not know that this method will scale with increasing task numbers, considering Neural Networks are already difficult and energy-time consuming enough to train as is.
That's what I'm getting at really: AIs that are expert/genius level at something niche and fall apart when applied to a similar task a human wouldn't have trouble adapting to. Once an AI is easily adaptable to many different domains without manual tuning people will be hard pressed to deny it is intelligent.
I don't think it is either. I'd want to see many more domains than is demonstrated in playing most games (e.g. conversation, object recognition, planning, maths)
IMO, we already have intelligent AI. It's just not intelligent the way we are used to dealing with. People don't want AI, they want a human brain in a box.
There was a recent blog post covering this and the difficulties involved:
http://togelius.blogspot.ca/2016/08/algorithms-that-select-w...
I also posted a link (https://arxiv.org/pdf/1604.00289v2.pdf) above which is an easily readable exposition on just how current approaches fall short. It's nothing so trivial as "it's just not what we're used to".
I take it you haven't seen the previous accomplishment of Deep Mind before they tackled Go. They used a Reinforcement Learning algorithm to play 50 Atari games - the same algo - with great results. They really created a generic learning algorithm.
But like I keep emphasizing, you can't take a neural net trained on space invaders and have it play Asteroids because each is a task specialized program that was the result of a search. While the search method is more general, the resulting program is not. You can use a single algorithm as simple as linear methods based reinforcement learning and get great results across a wide swathe of tasks but you can't claim to have found a universal learner.
Ah, I understand you now. I don't see the relevance though as I don't see why it's important how many algorithms, programs and computers is used to implement the AI. I imagine your cellphone is unable to do many things a regular human can do such as hold a basic conversion and learn to play new games which is why I wouldn't call it intelligent.
I am sure he is using different neurons for playing Go than for playing poker. His Go-related neural net is only able to play go.
Lee Sedol meanwhile is at least as capable as a Universal Turing Machine and was learning far more per game and modifying himself while also doing the highly complex tasks of vision integrated motion planning.
> Also, regarding other tasks: a neural net that recognizes cats can be easily made to recognize dogs too. A program that translates English to French can be made to translate other languages too.
I'm aware of these things. I was commenting on the shifting of the goal posts of what people call intelligent. In my opinion, when someone can deliver a concrete implementation of a computer that is competent (not even expert level) of many varied domains, then most people would call that intelligent. Right now, examples of AI are genius level at a niche domain and unable to do anything in all others (without tuning at least).
I don't think anyone would call an algorithm that in theory can apply to lots of domains intelligent. You need a concrete implementation to demonstrate this.
How will not applying the same technique/algorithm to lots of domains disqualify from being intelligent? RL and CNNs are being applied in many domains today. Is the complexity of the algorithm is what you're disagreeing with?
The human brain (if you strip out stuff that does not pertain to intelligence) can be seen as one super complex machine which can be encoded. We just don't know all the algorithm/code yet.
Very interesting how both your opinions differ. I had a similar discussion with someone on HN a while ago about this exact same thing. The discussion was very inferior to this one but you might be interested nevertheless I hope.
https://news.ycombinator.com/item?id=11939866
The other opinion I was referring to was argonaut's.
My original comment was about how people shift the goal posts about what AI is. My opinion is that in the same way you'd struggle to call a human intelligent if literally all they could do was play genius level Go, most people would not call a computer program intelligent if all it could do was play Go.
If the machine could play many other games and adapt to games it hasn't seen before that's more convincing, but you would expect an intelligent machine to be able to adapt to more varied tasks as well (e.g. having conversations, writing stories, doing maths, recognising objects). Right now, we have AIs that are genius level at one task that cannot even attempt other tasks e.g. genius level at Chess or even a whole category of games but couldn't have a basic conversation.
> How will not applying the same technique/algorithm to lots of domains disqualify from being intelligent? RL and CNNs are being applied in many domains today. Is the complexity of the algorithm is what you're disagreeing with?
I'd say it's not important if it's one algorithm, many algorithms, simple algorithms or complex algorithms, just that it's a single general purpose concrete implementation that is capable of doing many varied tasks and learning.
> The human brain (if you strip out stuff that does not pertain to intelligence) can be seen as one super complex machine which can be encoded. We just don't know all the algorithm/code yet.
Yeah, I don't think there's anything magical about the brain that a computer couldn't replicate in some form.
I totally agree with you on that one.
>Hmm, I think we're misunderstanding each other. Is your view that RL and CNN have been or can be applied to create a machine you'd call intelligent right now?
Not really. I was making a point to this:
>I don't think anyone would call an algorithm that in theory can apply to lots of domains intelligent. You need a concrete implementation to demonstrate this.
I was saying that even the human mind is one complex algorithm and we consider that intelligence. Something being understandable and us being able to apply to multiple domains can be intelligence as we see it today. It's just that we don't know what the complex algorithm is.
>If the machine could play many other games and adapt to games it hasn't seen before that's more convincing, but you would expect an intelligent machine to be able to adapt to more varied tasks as well (e.g. having conversations, writing stories, doing maths, recognising objects). Right now, we have AIs that are genius level at one task that cannot even attempt other tasks e.g. genius level at Chess or even a whole category of games but couldn't have a basic conversation.
This is exactly what I'm getting at. If we do find out the algorithm that effective replicates human intelligence, isn't that intelligence according to the definition of intelligence? If we find out the complex human algorithm, then we'd be able to do all the things that you just described
I think where our misunderstanding arises is that I'm saying you're disconnecting the working of human brain and algorithms. I'm saying human brain is one big algorithm and your statement :
>I don't think anyone would call an algorithm that in theory can apply to lots of domains intelligent
Wouldn't hold true in the case where we find out the algorithm for emulating the brain effectively. This disconnect in separating human intelligence from being anything other than an algorithm will make us see human brain as an un-understandable mystery box, which I think it is not.
Let's see, how many skills/domains can a particular human cover? For example, I can't speak Chinese. I haven't learned Chinese. Also, I have no idea about medicine. But people who learned medicine, know a great deal about it. Maybe a human can do 20-100 things, like walking, low level addition and multiplication, speaking a few languages, playing a few games and working in a few domains. Not an infinite list. In the same way RL systems and CNNs can be used for hundreds of different applications, depending on the data they are trained on.
Also, it is not the same neural net in the brain that handles two different skills, we use specialized neural nets for any of our skills too. If you stick together a few neural nets and a controller that selects the right one for the task, you could have a "single" system doing many things, just give it training data to learn those skills. Humans take 20 years to learn the necessary skills to function in society too. Neural nets can do it much faster and often surpass humans. DeepMind started Go a couple of years back and surpassed the best human player - how is it possible to do that, in such a short time span? And it wasn't a case of 'clever tricks' like the chess program Deep Blue.
Or even just talk about Go, answer complex questions about it in natural language, and state and prove interesting mathematical results about the complexity of Go.
I expect in 20 years undergraduate curricula evolve to cover NNs, and then those courses will have their salient bits adopted into mass market expository books/blog posts/code examples/etc. At that point someone like you will be saying something like you said about whatever the new hotness is.
Yep, we know, the consecrated expression for this point is "they just used clever tricks".
But at some point these clever tricks add up to something akin to imagination or intuition. Do you think humans are not made of "clever tricks" too? Could it be possible that we humans have the magic fairy dust of real intelligence sprinkled in our brains and machines are lesser than us?
If it's magic, it's human intelligence. If it's magic, it's consciousness. If it's understood, it's algorithms.
It's only a matter of time. There is no magic in science. Only in the application of it.
The "AI Effect" -- saying, "it's just an algorithm" once we succeed -- is an artifact of developing AI by using it as an algorithm for a domain-specific task. It is "just" an algorithm, because that's all it needs to be to win at go, identify cats, or whatever. Fundamentally, it's not very surprising that when you set out to make a system that's really good at playing go...you end up with a system that's really good at playing go. Of course it's hard -- that's what we should appreciate -- but most of the problems we consider "hard" are based on what we think it is hard for a computer to do in order to solve the problem. "Searching the space of all possible go moves is computationally intractable, therefore this problem is hard." In reality, the problem may not be as hard as we think if the agent doesn't have to search the space of all possible moves.
"Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function...
Minimize the informational free-energy of the product of the multiplicatively inverted brain-to-body reinforcement-learned energy function of reward with the brain-to-body reinforcement-learned energy function of punishment, via active inference?
Beh, those words are messy. The point is to talk about a Gibbs distribution with energy functions Reward(X, Y) and Punishment(X, Y), such that the "total" energy function is E(X, Y) \propto Punishment(X, Y) - Reward(X, Y). This then gives us a "goal distribution" for an active inference agent (like a human), defined "up to" the reinforcement-learned energy functions, whose limiting functions (need a lot of functional analysis and probabilistic reasoning over function spaces, there) are the "ground truth" causal relations by which the world causes reward signals through the body and its senses.
Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts, ex nihilo, rather than an efficient engine for distilling structure from experiences, and to be ontologically special to prove how special we humans are on a universal scale.
There are cognitive science and neuroscience labs hard at work on deciphering how actually-existing human thought works, but as far as I can tell, everyone facinated by "AI" is so enamoured with intelligence being special that they don't pay attention to those labs. A great deal of books and papers are thus published, and as far as "AI" is concerned, it's all in vain somehow.
I think we might have to step into borderline philosophical subjects like epistemology and sentience depending on who you talk to to define intelligence and whether you're the free will group or determinism group and so on.
I think you might agree with the statement that we can't yet come to a strict definition of what intelligence is, and once we have it, if we are going for intelligence or something more akin to stuff independent of intelligence like sentience and creative thought.
I'm personally in the determinist group and agree that there is nothing inherently magical about intelligence, just us being simpletons who cannot comprehend how a microprocessor works when you just give the latest intel chip. It is so complex that we need a lot of time and energy to comprehend it, but like the intel chip, I believe there is a design aspect to the human brain rather than it being a magical black box.
The problem is, the majority of us are the simpletons and have a long way to go before we have the knowledge that the intel chip makers have.
I apologize for the bad intel chip analogy, I probably could've come up with something better :)
The criteria should have always been "faster / more efficiently", "more accurately", or "both" in comparing AI to human intelligence for a given task or activity. By this measure, AI had enduring production-ready successes since at least the 1980s. Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count.
True AI is starting to look philosophically like a mirage, not the least because we may not ever have a strict meaningful definition for intelligence. What we consider intelligence varies, evolves, changes shape, consistency and predictability even if we consider just one of the various people we interact with on a given day.
As one of the latest, I think Brian Eno (https://news.ycombinator.com/item?id=12027055), like others before him who have formalized a similar approach to the matter of intelligence, is on the right track. This approach to deciding what constitutes True AI would do much to counter and eventually prevent the "AI effect".
Well, I agree that it is not definitive, but it definitely gives us some insight on where AI techniques are, comparing with the 'average human'. Now I know even the term average human doesn't have a strict definition, but the scale of intelligence[0] is so vast compared to the dumbest and smartest human, all that we need to know from that is if the technology is better than humans or not. If it is not, then I'll not care much, but if it is, then you've got my attention.
>Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count.
I think the problem lies with how we emulate intelligence. Will you be able to emulate me perfectly if you had not billions, but trillions of transistors? I'm assuming the answer is no, because, although we have all the power that the human brain (abstracting the neurons and other 'mechanical parts') has, we cannot simulate even a child's intelligence (now here I'm referring the ability to learn new things and communication, etc other abstract things etc, not computational ability). We need to know the technique just like we know have deep learning and are able to analyze pictures near-perfectly.
>True AI is starting to look philosophically like a mirage
Well, of course definitions vary. I think most people's (including mine) definition is that 'AI' (AGI, if you want to remove ambiguity) is surpassing human intelligence in the optimization area, like large scale optimizations like designing a spaceship, or curing cancer using techniques unknown to us right now.
I remember reading that and considering his view very interesting. With our (normal) perspective we don't really see AI like people see it centuries ago.
[0]: http://kruel.co/scale_of_intelligence.png - I apologize for the poor quality picture; it was a quick google image search
Now that very deep networks have become possible, and various graphical models and Bayesian approaches have also been folded under "deep learning" (for example, using back-propagation to learn complicated posterior distributions in variational Bayes) deep learning is not just about vanilla feedforward nets.
Still isn't and still masses of people go on to think that these "neural" networks work the same as neurons in a body do… while neuroscientists are still trying to understand how real neuronal networks operate with a bunch of pet phenomenological theories that most pretty much ignore physics despite the tools used lol
But for a neural net, you cannot say why this particular net should be trusted, as you don't know how it arrives at a solution. Therefore it's "scary" to use.
While I don't agree, it explains why it has been unpopular.
For training, a large set of decision trees are built randomly based on the input features.
When classifying input for one tree, each node considers feature value of the input, and decides on a branch. Leafs corresponds to a classification, so when a leaf is reached, the tree has classified the given input.
By having a large set of trees, and picking e.g. the most common resulting class (majority vote), we increase accuracy.
However, each individual tree can actually be reasoned about. E.g. you can see the analysis (nodes) leading to each class (leafs).
I've had some success with RDFs in the past, and highly recommend them!
They are very easy to implement, very efficient to train and query, and they seem to work really great on classification of "discrete input" (i.e. where the input feature values are binary or from relatively small sets).
Those are two different things. Vanishing gradient problems were ameliorated by switching from sigmoidal activation functions to rectified linear units or tanh activations, and also by dramatically reducing the amount of edges through which gradients propagate. The latter was accomplished through massive regularization to reduce the size of the parameter spaces: convolutional layers and dropout.
Stuff like residuals are still being invented in order to further banish gradient instability.
Is there something fundamental we are missing in going about building these deep learning stuff ?
Even a human brain has to train for ~4-5 months to become interested in shapes (https://en.wikipedia.org/wiki/Infant_visual_development). Once the human brain has been trained for these basic shapes for a while, it is able to quickly break down a new class (i.e. a cat) and recognize similar patterns in other images. This is something that is very similar to the way that training a deep NN works.
Also don't forget that the current NN are being trained mainly for photos, not moving images. A brain may recognize a cat by its tail-wagging or fur movements, which is a dimension that is completely missing from still images.
Also check out this similar post and discussion: https://news.ycombinator.com/item?id=9247851
Everyone who wonder how (on a superficial level) grown up humans are so good at learning new categories really should spend time around babies and toddlers and children for this reason...
You quickly realise how much training and brain development it actually takes before we're capable of doing much.
Then one day they start to get it (like "fire burns"), but it's still not there for sure until they experiment it deeply multiple times.
The dev in me can't help but see this two little humans as big mighty neural networks who spend their full uptime constantly ingesting tremendous amount of data and being restlessly tuned back by adults and experience :)
An untrained neural net has to learn everything from scratch (ha!), from pixel values to "knowing" what a cat is. There is some work on decreasing the amount of data needed to learn, but it's a very tricky subject.
> And how many different pictures of cats are there?
http://jmlr.org/proceedings/papers/v37/romera-paredes15.pdf
"Zero-shot learning consists in learning how to recognise new concepts by just having a description of them."
In that sense it's quite similar to a vision system of humans or other animals, which needs lots and lots and lots of early-age exposure to "learn how to see" (which is the hard part); and only after that it becomes possible to figure out cats of any kind just by seeing one or two.
Check out the recent paper "Building Machines That Learn and Think Like People":
https://arxiv.org/abs/1604.00289
Humans can see one or a few examples of a novel object, such as a cat, and create a fully 3D mental model of it. So we know what it will look like in different orientations and lighting conditions.
While in general I felt something similar that it has to be not just reams of data but also some sort of model (meta data) that when combined can produce innumerable combinations more easily.
I.e. we can recognise static 2D photos of cats, 2D movies of cats, 3D movies of cats, and real cats in the real world.
The invariants are probably relatively simple - a set of head geometries and head feature shapes/distribution, with some secondary colour and texture confirmation.
What's interesting is that we can recognise modifiers to the invariants - e.g. a shaved cat is still recognisably a cat, but parsed as "cat without fur."
We can also recognise invariants when they're pared down to essentials in cartoons and sketches.
https://www.youtube.com/watch?v=R9qdyXCVNVk
A lot of learning is really just data compression - finding a minimal set of low-resource invariant specifics from a wide range of noisy high-resource inputs.
A human child receives images on the retina at 20fps, say ... for 12 hours a day, for many years. That is a lot of training, a lot of images received by the brain.
What the kid does when we show it something new is to do a kind of fine-tuning of its neural net where previous visual experience is reused in order to quickly learn new types of objects. It's called one shot learning and it can be done in neural nets too.
That's the problem I always had, you may get them into a trained state, but good luck figuring out any reason 'why' they ended up in that state (or even what that state really is).
There are machine learning methods out there that are much better at explaining why, but fail hard on problems that DNN is good at.
IMHO, this is the nature of the problem, not the solution.
Can you give a specific example of what you mean? I ask because I see this sentiment often, but primarily from people who are very new to deep learning.
You can definitely debug a neural network. You mostly want to look at metrics from training such as gradient norms or try adjusting parameters to see if you can get a gain in overall performance as measured by cross validation performance.
You can definitely analyze a neural network. You do so by forming hypotheses, preparing datasets that reflect those hypotheses, and running them through your trained models, noting cross validation performance. It's also possible to visualize the weights in various ways, there are many papers written about it.
So what do you mean exactly when you say no one has figured out how to debug or analyze DNNs?
When it doesn't discover that it's a stop sign, how do you debug it? Did it recognize the shape.. who knows?
This paper does a good job of showing how CNNs learn a hierarchy of increasingly complex features to classify images: http://arxiv.org/abs/1311.2901
Barring other analytic tools (like looking at which parts contribute the most to the wrong result), the same way you test other things when you have a (somewhat) black box:
Form hypotheses and test them.
Maybe it didn't recognise the shape, so try adjusting the image to clean it up, and once you have one it recognises, try to reduce and alter the difference between them. Maybe it turns out the image e.g. has the stop sign slightly covered, making the shape look wrong, and there's nothing in the training set like that.
Maybe the hue or brightness is off and the training set is mostly all lit a certain way. Test it by adjusting hue and brightness of the test image and see if it gets recognised.
And so on.
There are plenty of other areas where we are similarly constrained from taking apart that which we're observing, so it's not like this isn't something scientists are dealing with all the time.
Within comp.sci. we're just spoiled in that so much of what we do can be easily instrumented, isolated and tested in ways that often lets us determine clear, specific root causes through analysis.
They tweak the input to maximize the response of specific neurons somewhere in the middle of the network to figure out what those neurons actually "learned".
Deep learning is, yes, a subcase of machine learning, and AI may be a circle within machine learning and enclose deep learning.
But truth be told, we will all regret the way we use the term AI now. Eventually the term AI will refer only to general intelligence (aka AGI).
However, you do have a point that simple machine learning aka linear regression is not really "Artificial Intelligence" if you raise the expectation of "Artificial Intelligence".
The fact that many laymen have a different conception of the term is hardly a reason to change all our uses of the term.
Machine learning is one of the but not the ONLY way to achieve AI. Artificial Intelligence is the objective, machine learning is one way to achieve it.
I think a (more) true AI would not need all this data and just be a _personal_ assistant, not needing all the big data of other people too. Maybe initially, once, and then be a good personal assistant, learning to know you like a human personal assistant.
It would need some common sense, which is now lacking, mostly.
"Deep Learning has enabled many practical applications of machine learning and by extension the overall field of AI."
Is it not the reverse - machine learning has enabled deep learning?
Can someone comment on how the two - machine learning and Deep learning relate? Is the relationship sequential i.e a data set from machine learning is the the input for a neural network? The diagram had the effect of confusing me.
"Machine Learning has enabled many practical applications of deep learning .."
No?
We are then shown a diagram by a computer scientist. Instead of cells and thunder, we see circles and arrows. Then we are told there is an algorithm that simulates what the brain does. Viola, we have our artificial neural network. Not only do they look similar, they have two words in common, neural and network!
And so for most of us, there is only one logical conclusion: It does what our brain does, so once our computers have the power our brains do, we'll have the singularity!
Of course, now we know this is complete bullshit.
Basically, computer scientists just took the names and those initial abstractions and ran with it. They never looked back at the biology or how brains actually work. The result is a ton of great research, but they've strayed further and further from neuroscience and from humans. Which is obvious, because they're staring at code and computers all day, not brain meat. If there is one thing AlphaGo proved it is that we've made a ton of progress in computation, but that it's a different direction. Just the fact that average people generally suck at Go should be enough to show that AlphaGo is not human (in many ways it's beyond human).
In the meantime, our neuroscientist have made progress also, except, they've done it staring at the actual brain. And now it's to the point where our brains look nothing like that original image our computer scientists were inspired with.
Now there is this (Harvard research): https://www.youtube.com/watch?v=8YM7-Od9Wr8
And this (MIT research): https://www.ted.com/talks/sebastian_seung?language=en
With advancement comes new vocabulary, and the new word this time is connectome.
Some incredibly smart computer scientists will, again, take the term and all the diagrams, and start programming based on it. The result will be Artifical Connectomes, and they will blow our socks off. Now, don't get me wrong. I am not trying to be sarcastic here. This is what _should_ happen. And with every iteration, we will get closer to AGI.
It's just that whenever I see articles about machine learning and neural networks, I can't help but think of that classic artist's rendition of neurons firing, and how it's basically complete bullshit. Like Bohr's atom, it's an illustration based on a theory, not reality. Now we have wave function diagrams and connectomes. But as a physicist would tell you, anyone caught with a Bohr's atom is stuck in the 20th century.
Neuromorphic computing is the field that tries to more accurately mimic spiking neurons, but making something useful out of it takes a backseat. It is still an open question if it is going to be useful.
It might be better to explain why deep learning is so effective, in clear language:
* Deep artificial neural networks are old, relatively simple combinations of math and code that are now able to produce accurate models through brute force because we have
1) vastly more computational power thanks to NVIDIA and distributed run-times; 2) much more data, and much larger labeled datasets thanks to people like Fei-Fei Li at Stanford; 3) better algorithms thanks to the work of Hinton, LeCun, Bengio, Ng, Schmidhuber and a raft of others.
Deep is a technical term. It refers to the number of layers through which data passes in a neural net; that is, the number of mathematical operations it is subjected to, and the number of times it is recombined with other inputs.
This recombination of inputs, moving deeper into the net, is the basis of feature hierarchy, which is another way of saying: we can cluster and classify data using more complex and abstract representations.
That clustering and classification is at the heart of what deep learning does. Another way to think about it is as machine perception. So the overarching narrative in AI is that we've moved from the symbolic rules engines of the chess victors to the interpretation of complex sensory information. For a long time, people would say AI could beat a 30-year-old at chess but couldn't beat a 3-year old at basic tasks. That's no longer true. We can go around beating 3-year-olds at name games all day. AI mind, beginner's mind.
But it's important to note that deep learning actually refers to other algorithms besides artificial neural networks. Deep reinforcement learning is one example. RL is also an old set of algorithms, which are goal-oriented. RL helps agents choose the right action in a given state to maximize rewards from the environment. Basically, they learn the function that converts actions to rewards given certain conditions, and that function is non-differentiable; that is, you can't learn it simply by backpropagating error, the way neural nets do.
Deep RL is important because the most amazing algorithms, like AlphaGo, are combining deep neural nets (recognize the state of the Go board) with RL (pick the move most likely to succeed) and other components like Monte Carlo Decision Trees (limit the state space we explore).
So we're moving beyond perception to algorithms that can make strategic decisions in increasingly complex environments.
We've written more about this, and implemented many of these algorithms:
http://deeplearning4j.org/ai-machinelearning-deeplearning.ht... http://deeplearning4j.org/reinforcementlearning.html http://github.com/deeplearning4j/rl4j