Top Artificial Intelligence system is as smart as a 4-year-old | Hacker News Reader

Top Artificial Intelligence system is as smart as a 4-year-old

computerworld.com

29 pointstokenadult53 comments

I should have expected to see a bunch of blather about AI overpromising and underdelivering. Nevermind the fact that Expert systems such as the descendants of Mycin and DART have been making billions of dollars in profits for decades when applied in specialized domains like logistics and health diagnostics. Nevermind the fact that we can't go a week on HN without seeing an article gushing about driverless cars. Nevermind the fact that mass production of everything from cars to toothbrushes is impossible/infeasible without it. Nevermind the fact that nearly all forms and modes of mass transportation are scheduled using algorithms developed in AI research. Nevermind the fact that you can actually use email, thanks to the AI research underpinning spam filtering. Nevermind the fact that computers have beat the best humans in some of the most well known games of strategy and trivia. Nevermind the fact that one of today's largest corporations, Google, makes nearly all of its profit from a single application of AI.

No. AI is just making a bunch of ridiculous promises that it never intends to keep...solely because general intelligence isn't f*ing Skynet yet and because Siri can't tell the difference between Ichiro and Itchy Euro.

The fact is ConceptNet does not claim to be a general intelligence, so comparing it to a four-year-old is comparing apples to oranges.

Be glad that we don't have AGI yet. The world would be a very different place with machines much smarter than us.

AGI - Artificial General Intelligence.

I'm sad because such a world is probably a better one, so long as the AGI is Friendly and doesn't turn us into paperclips or worse.

I don't know. It honestly sounds boring to live in a world where all our problems have been solved and there is nothing left to do or accomplish. Maybe we will all live in simulated realities, but that sounds kind of dystopic to me.

That sounds more like a failure of imagination than an argument that there will be no more fun or challenge or creative endeavors in the universe once the worst of our problems are solved. Do you really think that in a future where you no longer have to worry about death and taxes, and you have a superior brain which you can modify however you want, thus ending akrasia and unwanted depression among other things like having an even deeper sense of fun or an understanding of new problems we can't even fathom yet at our current development, that you will have less fun than you're having right now, today, reading HN?

You might like to read http://lesswrong.com/lw/xy/the_fun_theory_sequence/

What would be the point in doing anything? Anything you could build, program, or discover, could be done ten times better by an AI. It could probably even design entertainment, like movies, games, music, etc, far better than human artists.

A version of myself with vastly increased intelligence sounds interesting, but I'm not even sure it would be me. If you make enough modifications to your brain you become a completely different being. Different personality and thought process and little resemblance to who I am now. Which is disturbing to me at least. Along the thought of living inside of a computer doesn't exactly sound pleasant.

Even society itself might not exist anymore. Why would people interact with each other if there is nothing left to talk about that can't be instantly communicated? Why would people even spend time with each other if there are virtual realities to live in that are far more "fun". Spending eternity in a fantasy world sounds awful.

Yes I want to cure diseases and make everyone rich, but if you keep going, make a machine that solves every last minor problem and does everything there is to do, then there is nothing left for us.

And that's assuming that friendly AI is even possible, which I doubt it is, and I doubt will be discovered before AGI anyways. But either way the world we know will be utterly destroyed.

It's just a false clam and treated as such. Hacking a program that solves certain specific problems, given precise parameters and boundaries, like or better than a 4 year kid does not mean it has the intelligence of a being that routinely learns by himself, explores the world making up opinions about, expresses creativity and emotions.

This is just like claiming that Google Translate is more intelligent that the average American first grader because it can translate from Hebrew to Finnish better.

An autistic savant can learn extremely well given precise parameters and boundaries...and yet may never learn to tie his shoes or look both ways before crossing the street. But we still call them intelligent.

I think that gets to the core question of intelligence. I think that is merely a question of functional complexity. I am more intelligent than an ant because the mechanics of my decision-making process are far more complex and tuned. The question of "Is X more intelligent then Y?" is meaningless without narrowly defining the word intelligence.

Practically speaking, intelligence is just the ability to achieve your goals. An AI, ideally, would have some kind of utility function and make predictions about what actions will lead to the highest utility. The better the AI, the better it's ability to make predictions and get the outcomes it wants.

The amount of complexity is irrelevant. There are terribly complicated algorithms which do worse than simple ones.

I think part of it would be formulating goals. If you have a process that can very rapidly sum numbers, it ranks lower than one that is incapable of that speed, but can decide that summing numbers is a means to accomplish that goal.

Why would you need AI to mass produce toothbrushes ?

A friend of mine who knows Professor Robert Sloan

http://www.cs.uic.edu/Main/Faculty-Area

(I've met Professor Sloan directly once) told me about the article submitted here. He has a conference paper coming out for the AAAI conference next week,

http://www.aaai.org/Conferences/AAAI/aaai13.php

with more details about the research. It looks, from the first few comments submitted here, like many Hacker News readers think either

a) that the article wasn't sufficiently respectful of the field of artificial intelligence,

b) that the field of artificial intelligence deserves no respect.

But my understanding of Professor Sloan is that he takes artificial intelligence, his main current topic of computer science research, very seriously, and he is well aware of the societal importance of artificial intelligence research. Maybe the message was lost in the ComputerWorld reporter's treatment, but perhaps the conference paper will set the record straight.

On my part, I was glad our mutual friend told me about this article, as only today I completed an extensive edit of the Wikipedia article about IQ classification,

http://en.wikipedia.org/wiki/IQ_classification

so I've been pickling myself in scholarly writings on IQ testing recently. My friend is aware of that, and I think shared this link because of the angle of giving an IQ test designed for a human child (the WPPSI is strictly a preschool-age test) to an artificial intelligence system. Of course, we expect artificial intelligence systems to do different things from what preschool children do, so it's not surprising that an "expert" artificial intelligence system might not score high on a human IQ test. I'll read the professional publications when they come out to find out more.

Are any of you going to the AAAI meeting?

The article says the AI performed well on vocabulary, but poorly on comprehension, which resulted in "4 year old". I wonder how well it performed on each section separately? Was its comprehension nil? (as we'd expect)

Based on what I know about the WPPSI "comprehension" subtest, from taking a Wechsler Adult Intelligence Scales Revised (WAIS-R) test in the early 1990s, and from reading practitioners' manuals about IQ test administration since then, I would expert an expert AI system to correctly answer some but not all of the WPPSI comprehension subtest items. I wonder if the professional paper on this issue will provide more details of the system's subtest scores for each WPPSI subtest.

This is, of course, nonsensical claim. It might be able to recognize texts or make logical inferences like a 4 year old, but it is incapable of unassisted acquiring of knowledge just by exploring an environment, a task any for-year-olds routinely do most of the time.) In other words, systems could do some very specific and restricted task at the performance level comparable with a 4 y.o. But it is not "smartness".

http://en.wikipedia.org/wiki/AI_effect

I see what the article did there; switching from "real intelligence" to "intelligent behaviour". Not that "real intelligence" is well defined, sure; but that also means you can't just claim you achieved it nilly-willy.

Michael Kearns suggests that "people subconsciously are trying to preserve for themselves some special role in the universe". By discounting artificial intelligence people can continue to feel unique and special.

Right. Then why do I have no problem, and actually enjoy, noticing similarities between me and other animals or even plants? Even two electrons occupying different places at the same time could be argued to both be "unique and special".

What would make humans special in the face of even the most advanced AI is that we built it... and who, other than a lunatic or someone escaping from themselves, would look up to persons in a play he wrote? Don't mistake for jealousy what could also be explained with paying attention, that'd be my suggestion.

The article merely explains how perceptions are formed of AI research as a failure of capturing the essence of intelligence. Once an aspect of intelligence is understood, replicated, or improved on, it is no longer intelligent...it is reduced to mere computation. AI may not have captured the essence of human intelligence yet, but it seems to be doing pretty well with substantial aspects of it, given the fact that Humans have had a 200,000 year head start.

This isn't really the AI effect. Manually entering in facts and actually learning them from observations are very different things.

Machine learning can be defined as the machine augmented aquisition of knowledge through observation. It has been an integral part of AI research since 1957. The fact that you have separated Machine Learning from Artificial Intelligence is a perfect example of the AI effect.

AI, making absurd claims since 1956.

It is not AI, it is computerworld - very close to cosmopolitan.)

Ah, yes. "Five ways to give your hard drive the best defraging ever!"

AI has been saying things like this since it was founded.

1957 "there are now in the world machines that think, that learn, that create"

1961 "The processes of thinking can no longer be regarded as completely mysterious" in regards to the early General Problem Solver.

...

The list goes on and on.

This isn't to say that AI research isn't important, wonderful, creates enormous value to society (relational databases anyone) and has inherent value in the quest for knowledge, but such absurd claims have hurt AI more than helped it (note that absurd claims in physics have not had the same negative effect). See http://en.wikipedia.org/wiki/AI_winter

how many times have you read this about AI (quoting from the story now):

Basically, it's difficult to program common sense because scientists haven't yet figured out how to give systems knowledge about things that humans find obvious, like the fact that ice feels cold.

"All of us know a huge number of things," said Sloan.

So the story is essentially that a computer program was able to succeed at a pattern-recognition exercise, correct? Or am I missing something...?

> "All of us know a huge number of things," said Sloan. "As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don't appreciate having their tails pulled."

Then let the damned thing explore! Why do we make such huge advances in robotics if not for this? A baby-like robot shouldn't be that hard to make. I'm no AI expert but couldn't we just throw our best learning AI in there and give the thing a couple of years?

Why are we trying to short-circuit human learning instead of mimicking it?

This is what I want to see happen. I want to see categorical cross-referencing events/things where round is applied to ball and ball is applied to basketballs and baseballs. I want to see orange applied to basketballs, and then a connection is made between the fruit orange and a basketball. And just let that thing work, just leave it unassisted for a year as it grows more and more connections and see where the lexical processing and visual (color and shape matching) processing takes it.

Here's one team trying for that: http://www.inria.fr/en/teams/flowers

Ya I don't like how we are biased for machines being absolutely perfect or useless. If an AI system can't do something in a few hours that takes humans years to learn or think about, it's considered a failure. Humans spend many weeks, sometimes years on things, not to mention all the time up to that point they spent learning about the world or working on similar problems.

On the other hand most algorithms will quickly plateau and not make much progress afterwards. Machine learning algorithms are nowhere near the ability of human brains and letting them run for 4 years wouldn't be much different than running them for a few weeks. (Also time on a supercomputer isn't cheap.)

Leaving a robot to explore and learn wouldn't make it any good. If you look at the successes and failures of AI listed in the article, modern AI is well suited for sensory recognition and pattern matching, and it's bad at cognitive reasoning in the abstract - just as it has always been. There are logical reasoners, of course, but they can only reason about whatever the programmer has put previously in the system as inference rules.

Now letting a robot roam around the premises wouldn't be much different than feeding it with slides of sensory stimuli.

The kind of experiences that the bot could only learn from moving around are beyond its sense-making; it can't gain any significant advantage from having wheels and hands and chasing the environment, because it doesn't know how to use them to enhance its knowledge and learn new kinds of things, in the way that a toddler can.

The only approach I know that seems promising in that respect is IBM's Watson - I'm not sure how it works exactly, but even if it's doing basic pattern recognition, the huge scale of parallel process and the enormous data corpus that it's feeded into Watson ''might'' just be what it takes to achieve sense-making in an emergent way.

I would also argue that for intelligence to be comparable to a human, creativity must be measured accurately. Of course, this is something that classic IQ tests don't measure well, so maybe that's hoping for too much.

I totally agree. Lateral thinking is so much harder to program and architect than logical thinking - we don't even have a good operational definition for creativity that is rooted in physical space and in a conceptual framework.

What do you mean by creativity?

https://en.wikipedia.org/wiki/Creativity

To me, the mark of an artificial intelligence is general analogizing ability, and that's not what we have here.

A system capable of general analogy could potentially write its own drivers with some light scaffolding or guidance. A system capable of general analogy would be able to form causal models of our world, and to be sensitive to the differences between mere correlation and correlation with causal potential, just as rats and ravens do.

This system has not yet even tackled the intelligence of rats and ravens.

Skynet isn't evil; it's just over tired and needs a nap.

So basically it did like a 4 year old on the knowledge portion, and failed miserably on the intelligence portion?

I'm actually surprised that it only did like a 4 year on the knowledge part, I would have assumed computers would do better.

We aren't anywhere near AI, but AK (artificial knowledge) exists.

I'm not an expert in algorithms involving AI, but the ones I've been exposed share the characteristic of being brute force with giant if-then-else trees of possible decisions for the computer to make. It seems like any possible outcome a human can't for see will not be picked up by a computer either.

Which ones are you talking about? I have only a superficial exposure to them (mostly Expert Systems, Machine Learning, and Optimization contexts) but none of the ones I have seen come anything close to if-then-else trees. Okay...maybe Random Forests, but they are only if-then-else trees in the evaluation/prediction stage.

I meant search trees such as the ones used in minimax. I think of it as "if-then-else" in more of a colloquial sense, may be an incorrect description.

I read this as "Top Artificial Intelligence System is a Smart Ass", then noticed "a 4 year old".

I can guarantee you it wouldn't even come close to matching the natural language skills of a 4 year old. If so, that would be a major breakthrough for computational linguistics and NLP, and yet I haven't heard anything about that.

really, a 4-year-old? I thought even a cat wasn't possible yet (I remember a story about someone failing to make one as good as a cat). That's impressive, then.

I think that specific one was where someone simulated the same number of neurons that a cat had. However it failed because a virtual neuron isn't the same as a cat's and the training algorithms obviously weren't very similar.

Other AI projects vastly excel humans in some fields though (arguably chess is an example of this).

This link seems to focus on decisions making and the ability to appear normal in general. In this way I strongly agree with you. I've never seen an AI chatbot that could hold a conversation as well as your average 4 year old.

>Other AI projects vastly excel humans in some fields though (arguably chess is an example of this).

That would be relevant if the chess winning programs mimicked the way humans think -- so that the same intelligence could be transfered in other fields.

As it is, it's no big fit with regards to AI to win in chess by pruning decision trees and the like -- and it's mostly non transferable to regular reasoning.

It's like touting the fact that the computer can do 432342/4234234 million times faster than me as relevant to it being intelligent.

"That would be relevant if the chess winning programs mimicked the way humans think -- so that the same intelligence could be transfered in other fields."

If that were true for human intelligence, the Kasparovs and Carlsen's of the world could have lucrative side jobs solving e.g, protein folding problems.

I don't think it is as black and white as you claim it to be.

>If that were true for human intelligence, the Kasparovs and Carlsen's of the world could have lucrative side jobs solving e.g, protein folding problems.

Well, for one, great chess players are generally of higher IQ. So great chess and increased general intelligence ARE correlated.

Now, why should being good at chess also apply to "protein folding"? If anything, following my logic it would be the opposite: protein folding is more like number crunching than like the way human chess players think about moves. If humans played chess like computers, then yes, they would also be good at protein folding. But humans do differently. For one, the don't actually consider millions of possible future moves. They prune much more intuitively and effectively than AI chess engines.

You argued:

"That would be relevant if the chess winning programs mimicked the way humans think -- so that the same intelligence could be transfered in other fields."

That implies that, in humans, intelligence in chess can be transferred to other fields. I pointed out that that certainly isn't universally true. I think that correlation is fairly poor.

I also think the correlation between IQ and great chess playing is not that great, but don't have evidence for it. Chess playing _ability_, maybe, but good chess playing requires lots and lots of rote learning that the brightest humans might find too dull to spend their time on.

Also, one can argue that your claim that humans "prune much more intuitively and effectively than AI chess engines." isn't true anymore. Humans prune more, yes, but they also cannot beat today's best computers, so the 'effectively' part is up for discussion. Maybe, they are pruning too much? Or is it just that their evaluation function is inferior?

Agree. The last time I heard was it was as good as retarded cockroach.

Reply on news.ycombinator.com