Hacker News Reader

> My counter-hypothesis is this: Computers, compared to humans, will always be more data-hungry (i.e. worse at making general decisions without huge amounts of data) and more data-capable (i.e. better at making decisions with it).

I don't think this is necessarily true. Currently we're trying to push computers to do things that are relatively easy for humans to do. I would conjecture that these things are easy for us, not because we are amazing learning machines, but because we have millions of years of evolution, and years as infants with caring teachers going for us.

So, I think we are actually data efficient learning machines, just that we have strong priors, and we spend a lot of effort training each other.

If you compare humans to machines on problems which are not intuitive for us, I think you will find machines to be more data efficient than we are.

> I would conjecture that these things are easy for us, not because we are amazing learning machines, but because we have millions of years of evolution, and years as infants with caring teachers going for us.

And so do chimpanzees. Evolution must have provided us with something additional, which would be our rather more developed cognitive abilities to employ abstract reasoning and metaphor.

Those abilities aren't learned, they're innate, and they allow us to think in ways that don't require large amounts of data. An average human being can be shown an Atari game like Pacman, and easily understand what the objective of the game is almost right away.

Well this is silly. Show a toddler that game and they'll have no idea what the purpose is or even why its a game.

Humans draw on massive reservoirs of knowledge to comprehend anything.

That's true, but the question is whether you can train ML on a massive reservoir of knowledge, with the result being a similar understanding of the world that humans possess.

There is a long term attempt to give machines a common sense understanding of the world by specifying several million rules. That's the Cyc project.

> And so do chimpanzees. Evolution must have provided us with something additional, which would be our rather more developed cognitive abilities to employ abstract reasoning and metaphor.

If you have an evolutionary learning algorithm you don't expect every branch to be equally capable, it doesn't mean we didn't get here along the same path.

I basically agree that humans have some "innate" ability, but this innate ability exists as the result of an evolutionary process.

> Those abilities aren't learned, they're innate, and they allow us to think in ways that don't require large amounts of data. An average human being can be shown an Atari game like Pacman, and easily understand what the objective of the game is almost right away.

Taking the single experience with a single Atari game is sort of missing the point, which is all the time we spent learning up until that point and all the evolution that went on until that point.

It also sort of misses the point that Atari games are explicitly designed to be understandable easily by humans, they're not something that just appeared and we happened to be good at it.

> I would conjecture that these things are easy for us, not because we are amazing learning machines, but because we have millions of years of evolution, and years as infants with caring teachers going for us.

But if the argument is that AlphaGo is the right approach to creating an AGI, then we should at some point expect it to learn how to recognize the goal of various tasks without a huge amount of training.

Maybe evolution provided us with something additional that is lacking in current generation of DL. And there are AI researchers who think that you need ontologies for the machines to understand the world, and that it's not reasonable to expect a machine to be able to learn everything from scratch, because the world is too complex for that.

It's not reasonable to expect AlphaGo to replay evolution in order to gain the ability to do abstract reasoning.

> But if the argument is that AlphaGo is the right approach to creating an AGI, then we should at some point expect it to learn how to recognize the goal of various tasks without a huge amount of training.

I never said anything about AlphaGo or AGI. I said that humans are not as good at generalizing from few examples as people would like to believe.

But a game like Pac-Man is intuitively understood by the average human because it’s fundamentally a “human” game - designed by humans, gives humans dopamine and other chemical hits in a way that we might even perceive as “fun”. Imagine a game that requires lots of computation and no human-friendly interface - a machine would obviously “learn” the rules a lot faster.

The more evidence we uncover, especially the research around Alpha Go Zero (self play with a specific objective in lieu of millions of years to develop keen general intuition) the more it feels like “human-like” intelligence is not some incredible holy grail of general intelligence but an emergent property of any reasonably directed algorithm.

Another random thought, but cro magnon man comes to mind as a human-like intelligence that proved simply not as cunning or vicious as human intellect and was outcompeted and stamped out. Imagine if we were to discover the AI equivalent of cro magnon intelligence - would we be quick to dismiss it as subpar and “not general enough” even though it emerged through the same algorithm (natural selection)?

> But a game like Pac-Man is intuitively understood by the average human because it’s fundamentally a “human” game - designed by humans, gives humans dopamine and other chemical hits in a way that we might even perceive as “fun”. Imagine a game that requires lots of computation and no human-friendly interface - a machine would obviously “learn” the rules a lot faster.

But if we're talking about creating AGI and the concerns that go with that (full automation, self-directed goals in the real world, etc), then the question is whether DL is enough on it's own to get there.

As such, comparing AlphaGo to humans on a variety of tasks like Atari Games or Go is kind of the point. And Google's goal is turn it into a product, which means doing tasks humans currently do.

> And so do chimpanzees. Evolution must have provided us with something additional, which would be our rather more developed cognitive abilities to employ abstract reasoning and metaphor.

Note that the current capabilities of AI systems are nowhere near the general capabilities of a chimpanzee. It seems reasonable to assume that the hard task is to come up with the prior of the mammalian brain. The "easy" part is to discover the parameter space on top of that prior, be it chimpanzee or human.

You speak as if a human being shown Pacman for the first time and figuring out the controls doesn't require a ton of data. There's so much data that goes into that. You have tens of billions of photons entering your eye during that playtime. This is sensory information that your brain decodes and reintegrates on the fly, then relays to the required parts of your body.

Could you have experienced all of that without seeing Pacman? Probably not. Once you have seen and played Pacman enough, you can probably imagine an entire instance in your mind. That's because we are good at storing and retrieving certain kinds of data. The data was required in the first place, though.

It seems reasonable to me to believe that there are multiple ways to be "intelligent", and that different kinds of intelligences will excel at different tasks. When we think of "general intelligence" I think we default to thinking about "human intelligence" simply because it's the best example of any kind of general intelligence that we have access to. But I don't see any reason, in principle, to think that "machine intelligence", perhaps in the "data-cruncher" style, can't ultimately exceed human style intelligence.

I mean, we already know machines can be "smarter" than humans in narrow domains (Chess, Go, Checkers, calculating square roots, calculating integrals, etc.) so if we find a way to combine that with some kind of "generality", Bob's yer uncle.

I strongly agree. We many never see a machine that can fool humans into think that machine is also human, but then again why do we actually want that? Just for novelty?

We look at a dolphin and we can say "that is an independently intelligent animal" that we can't really do much with. We look at a dog and say "that is a useful, trainable, intelligent animal" that we as humans couldn't have survived without during parts of the history of our species. A dolphin is far smarter, but it doesn't matter because there's only one intelligent animal we couldn't have lived without, and it's actually pretty dumb compared to a dolphin.

The question is, do we want an AI that is smart by itself, or do we want an AI that is smart and useful to us? Those don't have to be the same thing, as evidenced by the dolphin vs the dog.

Humans are really, really good at producing very efficient machines with strong intellect, we do it by accident all the time. We don't need more humans, humans are flawed and humans have a lot of undesirable traits to go along with the intelligence. Strong, general AI will be its own species, with its own unique way of thinking and its own quirks. Trying to replicate humans exactly is futile and worthless.

We want a machine that can learn and do useful stuff. We don't need a human made of silicon for that. We need a mechanical dog.

Likely some fear a sufficiently intelligent AI without human like values will render human life obsolete-- and then extinct

Humans seem pretty good at enacting the latter all on their lonesome

Strong, general AI will be its own species, with its own unique way of thinking and its own quirks. Trying to replicate humans exactly is futile and worthless.

Well said.

We don't need a human made of silicon for that. We need a mechanical dog.

That's a very eloquent way of putting it. I may have to steal this quote from you sometime!

They are smart just as water is smart in finding the path of least resistance.

> My counter-hypothesis is this: Computers, compared to humans, will always be more data-hungry (i.e. worse at making general decisions without huge amounts of data) and more data-capable (i.e. better at making decisions with it).

Counterexample:

AlphaGo Zero is currently the best Go player in the world. (It beat AlphaGo, and AlphaGo beat humans.) AlphaGo Zero learned to play Go entirely by playing itself; it was given no training set at all.

It's still more data-hungry, though. AlphaGo can play many millions of rounds against itself in the time it takes a human to play one round.

Humans are many magnitudes more efficient at improving a skill, computers just appear to do it better sometimes because they can move faster.

>Humans are many magnitudes more efficient at improving a skill

In the case of go, most definitely not. AlphaGo Zero achieved world-beating performance in just three days, running on four TPUs. Becoming a world class player cost somewhere in the region of 150kWh.

A fairly sedentary human requires about 2.5kWh of food per day. Achieving mastery of go takes at least 10 years of full-time study, so we're looking at a bare minimum of ~9000kWh. That excludes the energy inputs to make that food (often orders of magnitude higher) and the multitude of other energy inputs required to keep a human being healthy and sane.

> capable of making decisions without billions of data points, just like humans do.

Doesn't this ignore transfer learning? Humans have orders of magnitude more than billions of data points over their lifetimes.

Additionally, we have absurd amounts of information baked into our genes that give us big head starts on network architectures, motion, vision, etc.

Yeah, compressed using zip there's about 50MB difference between a bacteria and a human... at least kurzweil said something like that ;D

The genes have no information about the world, just information on how to form a nervous system.

Computers get billions of data points for a single topic/task. Humans get exposed to ginormous amounts of data that's less focused. The amount of sensory information we are taking in is incredible. That information isn't focused on a single task, but we can apply learnings from one field and apply it to another. If a human is trained to sort out foul fruit the human already knows how to generally differentiate between different objects they are looking at, smelling etc. They already know about apples; they already knows that fruit can spoil. It's just combining existing knowledge and skills. On a more complicated task like learning a new language the advantages are similar. I'd bet that a sufficiently large, neutral network that already knows how to perform many tasks would be faster at learning new ones as well.

> The further we get into the future, the more I think, what if our data-crunching approach to AI is simply the best thing there is?

Technically speaking, as long as we equate "AI" with "machine learning", it's a downright trivial statement. The important thing in ML isn't just the presence of "data-crunching", it's the quantity of training data relative to the size and complexity of the hypothesis class.

As long as "intelligence" requires dealing with ambiguous sensorimotor data, it will involve some statistical component, and will therefore involve a "data-crunching approach" somewhere in it.

>But these foretold "general" intelligences, who don't need excessive data,

Hierarchical generative models already do phenomenally well at one-shot and small-sample learning against basically all previous ML methods. Yes, this includes deep learning.

+1 I agree with you. Even though friends like Ben Goertzel believe and work hard to create AGI, I think short and medium term the path to much better AI will be in assistive systems. I manage a small machine learning team at a large bank and I am an all-in believer that systems built with deep learning, probabilistic graph models, <fill in any master algorithm you want here>, etc. will fundamentally change the way knowledge workers work and transform society. This belief makes me excited to go into work every morning.

I love Douglas Hofstadter’s work and I think I own all of his books. I am not very academic in my outlook. I love technology for what it lets me build. Reading Hofstadter is like looking into the mind of someone with a very different world view from my own.

You're still extremely optimistic about the 'intelligence' of state of art data driven approaches, even if they aren't general intelligence. I'm not sure where that optimism is coming from.

The chess example... "They were wrong about AI never beating grand masters, they're going to be wrong about X". Well, if you make the board bigger, there won't be an AI system that can beat a human.

Play Dota 2, but introduce a random variable that can't be known beforehand by anyone, like things in the real world, and the AI will always be beatable.

Great for specific domains, obviously, but your optimism about doing more advanced stuff, perhaps doesn't seem so grounded.

> Play Dota 2, but introduce a random variable that can't be known beforehand by anyone, like things in the real world, and the AI will always be beatable.

I wonder about a board game that randomizes the rules in simple ways. A human could understand the rule changes and adapt. To what extent can software be trained to do that?

Sounds like you're talking about general game playing[0], where a computer is programmed to take, as input, the rules to the game, and then compete. Looks like competitions are against other computers, but this isn't an area I'd expect humans to dominate in, long-term.

[0]https://en.wikipedia.org/wiki/General_game_playing

People have already put work into finding chess-like games, or variants of chess, that humans can play well (especially if they have some familiarity with chess) but that computers will struggle with.

Arimaa -- where computers did eventually reach the point of defeating humans -- is an example of this. Arimaa tried to attack both "opening books" and move-tree searches, by allowing the initial position to vary every game and by having each turn consist of up to four individual moves by potentially multiple pieces. The official challenges also required that computer Arimaa systems run on commodity hardware, and did not allow for modifying the computer "player" in between games of a challenge. It got through twelve yearly human-versus-computer challenges before the humans finally lost.

He did write that about chess, but clearly labeled as a personal hunch.

Your own hunch about what’s unimaginably far off differs from mine, fwiw — I’m very unsure but would not bet against human-flexible AI in our lifetime.

It's as if when considering a supersonic jet airplane, we were to ask, "When will it be able to power itself by catching fuel in flight?"

After all, some birds can do that, so we know it must be possible.

Turing test is actually super easy to beat given enough resources. Just have enough data and imitate responses.

Chat bots already can convince humans that they are human. Not reliably, some people can still tell the difference or ask tricky questions.

There are only so many ways to tell if someone you're talking to is a bot. Bots can already spit out meaningful sentences.

It's the person who is testing the bot that's the limitation. They need to have VERY good intuition about when it's a person being silly or a bot that can't quite find the right response. If you read the Wikipedia article on the Turing test, you can see that computers have already been able to pass it.