None of those techniques are new, nor are they fueling the AI hype cycle.
I purposely conflated AI with "deep learning" because it is the source of the hype. And in reality, what most AI startups claim to be using.
What is new is the hype around deep learning that took off after 2012, and because Google and Facebook decided to champion it.
In any case, as far as I can tell "AI today" is anything that is "AI" and that exists "today". How do you mean "AI today"?
>> I purposely conflated AI with "deep learning" because it is the source of the hype.
I don't understand why you would do that. You are aware that there is hype and that it is increased by misuse of the term AI. And you purposefully misuse the term AI in a way that increases the hype? Why?
You are completely missing the point: "deep learning" and AI are mostly synonymous in the current hype cycle, and it began with breakthroughs in deep learning.
How do you mean "AI today"
The AI industry that I work in.
Can you please edit swipes like that out of your comments here? They tend to degrade discussion. If you simply provide correct information, your comments will be stronger and their effect on the thread at large more salutary.
As a third party, may I suggest that this statement amounts to a 'swipe'?
But I've seen the same statement used in a derogatory manner in the sense of "get this done, it's not difficult [if you're not an idiot]" by a bad manager or two. To me it does carry that other sense.
So could we please all not throw about big proclamations about what "AI today" is ("just" or not), without first making sure that we have a thorough understanding of what we are saying?
I thank us all in advance
Edit: Sorry, I see you are the OP in the thread. Note that I did not aim that specifically at you and I included myself in "us". I understand you may have felt frustrated that I challenged your knowledge of AI but I sincerely think that you could have researched the subject a bit better before stating what you think it is.
Edit II: At the very least you could have tried to talk a bit more about why you think that "AI today is just capable of curve fitting". Making such a strong statement without any attempt to back it up with some kind of explanation (I'm not saying you should reference sources and bring "evidence" or anything, just explain it) comes across as a bit, well, ill-informed. With respect.
You just defended a swipe using another swipe.
Your argument is rooted in terminology. Yes, in academia, AI means more than deep learning. Practically speaking, AI and deep learning are synonymous in startup land. And yes, supervised NN techniques are just curve fitting, and are not practical for program synthesis. Which was the original subject of this monotonous thread.
I agree that this thread is dragging on a bit, but it started with a very bold proclamation expressed in strident language criticising peoples' apparent ignorance of the subject- by yourself: "Good grief" and "When will people wake up" rather set the tone of your comment. If you choose to open a conversation like that, with a broadside against "peoples'" ill-informed views I would expect you are prepared to take a bit of criticism regarding the lack of depth of your own views. If not and my criticism has upset you, I apologise, but in that case, maybe you can try to be less provocative in how you express your views in the future, because provocativeness tends to elicit robust reactions.
Edit: In any case I just wanted to say: I get that you're annoyed by our conversation but I'd like to thank you for keeping it civil (if a bit tense) and not resorting to personal attacks. Cheers.
If someone's comments seem ignorant or under-researched, the way to address that is not to put them down, however mildly, but to add correct information about the topic. This has the bonus effect that, in the case where they actually do know a lot about the topic but just have a very different view of it, you won't inadvertently insult them. Also, it's worth remembering that if X is the topic, then "the level of someone's knowledge about X" is actually already a step off topic. Stepping off topic can be great when the step is in a curious direction, but definitely not when it's in a provocative direction.
I'm actually annoyed at myself about this, so I'm definitely trying to remember to be more careful in my comments. Your level-headed moderation is a great help in that, thanks.
I have to say something though- curiosity is only one side of the coin (the coin being the pursuit of knowledge, I guess). The other half is passion. Passion is what causes heated debate, but it's also what causes people to debate in the first place. I think it's a hard balance to strike and we will all need an adult in the room, to help focus our conversations, for a long time to come. Probably not what you want to hear though :)
It's easy to understand how a mild swipe provokes a more aggressive one; indeed it's hard to resist being carried by that current, but that's just what the site guidelines ask us all to do: https://news.ycombinator.com/newsguidelines.html.
AI is most certainly not synonymous with deep learning. It is just people in the industry who do not know anything about AI and who recently jumped on the deep learning bandwagon, who think they are, and people in the tech press who don't have the time to do proper research. I don't see why we need to perpetuate their misconceptions.
Actually, we don't.
Yes, I suppose this is not the best way to say what I wanted to say without getting peoples' back up. I'm leaving it as it is since it's already been read a few times from what I can tell, but here's a less rash version.
What I mean is that, because of the tremendous recent success and public exposure of deep learning, many people have become interested in it who do not have a background in AI, or even in computer science, and who therefore enter the field with big gaps in their understanding of what "AI" means. That is my experience anyway.
Well, it's a shame to work in a field and not understand its history, not least the history of what has already been achieved and what has failed, and how, so that one does not have to repeat history. So it's in everyone's interest to avoid making statements with great certainty when this certainty is not backed up by long-term knowledge.
For the record, I'm a newcome to the field myself. But I have a background in classic AI, specifically logic programming, so I do know the long story.
The war for terminology is more lost than differentiating "Hacker" from "Cracker" when referring to computer security. There's a specialist arena where the distinction is occasionally respected. This is not that forum.
Artificial General Intelligence is so far off that it's not a general conversation topic, it's not even a specialized conversation topic - It's a fantasy conversation topic.
I think "Hacker News" is exactly that forum.
Have you seen what neural nets are now capable of? Speech synthesis/transcription, voice synthesis, image synthesis/labeling/infill, style transfer, music synthesis, and a host of other classes of optimization problems which have intractable explicit programmitic solutions.
The hype is justified, because ML has finally arrived, thanks primarily to hardware, and secondarily to the wealth of modern open research, heavily influenced congregations of leading researchers enabled by funding at Google, Facebook, etc.
The problems being solved by "curve fitting" ML were simply unsolvable by any practical, generalizable means before recently, and the revolution is just getting started.
I have also seen what neural nets are incapable of. Specifically, generalisation and reasoning. Says François Chollet of Keras [2].
AI, i.e. the sub-field of computer science research that is called "AI" and that consists of conferences such as AAAI, IJCAI, NeurIPS, etc, and assorted journals, cannot progress on the back of a couple of neural net architectures incapable of generalisation and reasoning. We had reasoning down pat in the '80s. Eventually, the hype cycle will end, the Next Big Thing™ will come around and the hype cycle will start all over again. It's the nature of revolutions, see?
So hold your horses. Deep learning is much more useful for AI researchers who want to publish a paper in one of the big AI conferences, and to the FANG companies who have huge data and compute, than it is to anyone else. Anyone else who wants to do AI will need to wait their turn and hope something else comes around that has reasonable requirements to use, and scales well. Just as the original article suggests.
_________________
[1] http://techjaw.com/2015/06/07/geoffrey-hinton-deep-learning-...
Geoffrey Hinton: I think it’s mainly because of the amount of computation
and the amount of data now around but it’s also partly because there have
been some technical improvements in the algorithms. Particularly in the
algorithms for doing unsupervised learning where you’re not told what the
right answer is but the main thing is the computation and the amount of
data.
Say, for instance, that you could assemble a dataset of hundreds of
thousands—even millions—of English language descriptions of the features of
a software product, as written by a product manager, as well as the
corresponding source code developed by a team of engineers to meet these
requirements. Even with this data, you could not train a deep learning model
to simply read a product description and generate the appropriate codebase.
That's just one example among many. In general, anything that requires
reasoning—like programming, or applying the scientific method—long-term
planning, and algorithmic-like data manipulation, is out of reach for deep
learning models, no matter how much data you throw at them. Even learning a
sorting algorithm with a deep neural network is tremendously difficult.We had something, but if we really had reasoning "down pat", we would not now be reading an article about someone faking automated app development. Programming is all about reasoning.
That is relevant to your comment. The work on automated reasoning (or "inference", etc) really started in the '50s with Church and Turing, then reached a peak in the late 80's and 90's with work on automated theorem proving (there was a great big push at the time to solve very hard problems to do with the soundness and completeness of inference procedures, particularly resolution) and is still going on (for example with Constraint Programming and Answer Set Programming etc). The result of this work was logic programming. I'm leaving out all the work on functional programming that was just another branch of the same tree, if you like, because I don't know it that well but I'm sure others on this board can complete the picture. Then of course there was all the other classical AI stuff on planning, grammar learning, game playing etc etc that you can read about in Russel & Norvig.
Now, all this work could potentially be turned to the task of automated programming- but automated programming was never the goal of all that research. There was a lot of work on program synthesis, but that was just another AI sub-field with its own specific goals, that were not the overarching goals of the field as a whole. That is why we don't have automated programming at the push of a button, today: because it was never the main subject of AI research.
Edit: bit of a plug. Like I say in another comment, my PhD is on algorithms that learn logic programs from examples and background knowledge (both of which are also logic programs). That's Inductive Logic Porgramming. Our stuff works. We can learn recursive programs and even invent sub-programs that are necessary to complete a programming task and that are not provided by the user. We are making big leaps all the time and we're way, way ahead of neural program synthesis and the like. There's also a whole field of Inductive Functional Programming that does the same stuff but with functional programming languages. Automating app development with that sort of technique is mainly a matter of engineering- the research is out there. But, you haven't heard anything about it because the hullaballoo about deep learning is covering everything else up and most people don't even know there is AI outside of deep learning. Hence my comments in this thread (rather obviously).
All the stuff you talk about is rigid and omits the thing that makes human "reasoning" valuable - context switching.
I think technical people are often blind to this because they don't do very much of it themselves, but it's the fundamental thing that makes people different from machines, and complementary.
I'm not sure what you mean by "context switching" but I will agree with you that following rigid inference rules is not how most people think most of the time. However, we do have the ability to think in this way and this way of thinking is very useful for certain problems where we can't just intuitively come up with a good solution. For instance, scientific thinking is of this kind.
Historically what's really been missing from most attempts at simulating human reasoning is "common sense"- background knowledge about the way the world works. If we could successfully encode even a ten-year old's worldly knowledge, we could probably build an automated reasoning system that would appear much smarter than a ten-year old, by dint of it being a) much faster, b) much more accurate and c) much more, well, logical. But, we have so far failed to instill common sense to our programs and models so they remain at best idiot savants; if not simply idiots :/
I don't think that's the only reason at all. I think it's a much harder problem than you're giving it credit for. Great to hear that you're working on it, though.
So perhaps my bad for using the turn of phrase "at the push of a button"- it suggests more automation than what I have in mind.
When I'm done with my PhD I might even consider launching a product :)