This appears to be the pattern of AI development overall: booms and busts of hype and disillusionment that leave behind useful tools that most don’t consciously consider to be remnants of a previous AI bubble.
I went through Stanford CS in the mid 1980s, just as it was becoming clear that expert systems didn't really do much. There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. Big-time denial as the field collapsed. Almost all the 1980s AI startups went bust. The "AI winter" followed. Today, expert systems are almost forgotten.
This time around, there are large, successful industries using machine learning. Tens of thousands of people understand it. It's still not "strong AI", but it's useful and profitable. So work will continue.
We're still at least one big idea short of "strong AI". A place to look for it is in "common sense", narrowly defined as "not doing something really bad in the next 30 seconds". This just requires lower mammal level AI, not human level. Something with the motion planning and survival capabilities of a squirrel, for example, would be a big advance.
(I once had a conversation with Rod Brooks about this. He'd been building six-legged robot insects, and was giving a talk about how his group was making the jump to human level AI, with a project called "Cog". I asked why such a big jump? Why not try for a robot mouse, which might be within reach. He said "I don't want to go down in history as the man who created the world's best robot mouse". Cog was a flop, and Brooks goes down in history as the the creator of the robot vacuum cleaner.)
That would be Ed Feigenbaum, the man who (IMHO) almost single-handedly brought on the AI winter of the 80s. Because of him we had to call all the AI research we did in the 90s something other than "AI" lest it get shut down instantly.
He was taken seriously at the time. Chief Scientist of the USAF at one point. Turing Award.
[1] https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw46...
Isn't this pretty much the current opinion of the majority of the thousands of AI researchers and programmers today? Maybe this guy was early to the party but his vision seems in alignment with today's practitioners.
The reason the hype of today is more dangerous is what I call the grandmother issue: In the 80s we dreamed of a program that would recognize your grandmother when she walked in the room. That was one of our holy grails.
Good news: The grandmother recognizer was finally built with deep neural nets in the 2000s and it works pretty well. It's not perfect but if you tell it to play grandma's favorite song when she walks in the room it's no big deal if it occasionally fails to recognize her, or if it plays the song when someone other than your grandma walks in the room.
Bad news: We now have people who (effectively) want to attach the grandmother recognizer to a machine gun with instructions to shoot your grandmother and only your grandmother, and never to fail to shoot her if she walks in the room. Suddenly the Type 1 and Type 2 errors are a whole lot more consequential. Modern NNs are simply not fit for that job.
We have pretty good AI for low-consequence purposes, but it cannot be used for high-consequence purposes without a lot more fundamental research. Incremental improvements to deep learning are not going to get us there.
I think you're underestimating how smart a squirrel is.
(John Rober’s squirrel maze, one of the most wonderful concotions I’ve found on Youtube.)
The first of these questions has led to real progress in things like image recognition, whereas the second of these questions has not led any real progress in digitizing squirrel brains.
So true bottom-up is not hopeless.
Would that be the "robot vacuum cleaner that can't see shite?":
https://www.boredpanda.com/robot-vacuum-cleaner-spreads-dog-...
We now have protein folders and superhuman Go players -- that's new.
I agree that ML ("AI") is currently at the alchemy stage. And, guess what? A neural network isn't even Turing complete! [citation needed - correct me if I am wrong.] So ML can only compute SOME functions.
AGI, when it comes, and believe me, it will, will have ML as part of its structure, but only a small part.
But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.
It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall.
Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam.
Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.
This seems to me quite a deep insight. But how would you formally define a process that is gradient-based?
We're making some progress on the convexity bit (heat functions, RL, etc.), but yes, there are other areas of statistical research all involved in trying to solve those sorts of problems.
ML is not necessarily a panacea, but just because you can point to problems that it doesn't solve doesn't mean it has no "real accomplishments."
> DL, like human cognition, are NOT driven by gradients.
???
1. Self-driving cars. The ultimate being no steering wheel and carefully software controlled and permissioned as to where you are allowed to go.
2. AI powered Body monitoring devices (e.g Fitbit) that can be used to biologically monitor millions of consumers body states and functions.
3. Any new kind of surveillance technology. Miracle mass surveillance is something that's really interesting to this market.
4. Large centralized social media networks that aggressively moderate content with AI.
5. Various kinds of transhumanist biotech that design all sorts of biotech stuff with AI. What the heck is Calico up to anyway?
6. Blockchain stuff. AI enabled or otherwise.
They are also interested in environment tech like any kind of alternative energy, no matter how speculative or impractical and fake meat, but I digress.
So I would guess the bubble is the WEF crowd and friends with absurd amounts of money trying to make their future, for good or ill, a reality and investing without much regard for the economics of the project.
[1]https://www.weforum.org/agenda/2016/01/what-is-the-fourth-in...
speech recognition hype... now most phone trees use them.
electric car hype... now california is full of them.
self driving cars... now many new cars have driver assist.
Electric cars are NOT AI!
But it is true that "california is full of them", whatever "them" may be. That may have something to do with the crowd (whores and madmen) attracted by the California Gold Rush and whose descendants now mine their wealth in VC endeavors, including deep learning.
https://en.wikipedia.org/wiki/California_Gold_Rush
"driver assist" is a long way from a "self driving car".
I'm saying AI is a technology that is being overestimated now. People are trying to use it for everything now. I think in the future when the hype has worn off it will not have worked for everything hyped. But it will have worked well for a couple things to the point they are taken for granted or have become invisible.
I know I'm paraphrasing someone smarter than me there.
(Edit: Larry Tesler maybe)
But then someone builds an agent that can do X, but it can't do anything else. So people don't view it as intelligent, since it doesn't do any of all the other things people associated with being able to do X.
So it has nothing to do with being used to computers solving new problems, it is that we now realize that solving those problems doesn't really require the level of intelligence we thought it would. Of course solving new problems is great, but it isn't nearly as exciting as being able to create intelligent agents.
Edit: And it isn't just hobbyists making this mistake, you see big AI researchers often make it as well and they have for a long time. You will find tons of articles of AI researchers saying something along the lines of "Now we solved X, therefore solving A, B, C, D, E, F is just around the corner!", but they basically never deliver on that. Intelligence was harder than they thought.
I'm still on the bandwagon that AGI is already solved, and we just don't recognize it because we aren't as complex or magical as we like to pretend we are. We are just currently selected-for species survival, and not even in the present environment as the dominant species of the planet. That doesn't make us "Generally intelligent" any more than many other systems on the planet (including the computational ones). I'm not even convinced that "intelligence" is especially a thing.
If we're trying to replicate "human intelligence," then I think that's something else entirely, and would require removing a lot of capability from the AI that I don't think anybody would want to remove from AI, given that we have plenty of serviceable humans to do the work of human-intelligence style tasks.
No, when folks say AGI, they mean "cares about the things I care about, and is more accurate, but can speak in a way I understand." But how is a network of neurons going to care about what you care about if they see the world in a different way, or have been trained on solving a different set of problems than "living a life as a human?" It just doesn't make sense to expect, or even want, human intelligence from a synthetic intelligence.