Hacker News Reader

Comment by pavlov | Hacker News Reader

This suggests a thought experiment for those worried about superhuman AI. If you add more matchboxes to this setup while teaching it to play something else, is it going to become intelligent eventually? Maybe at a billion matchboxes?

If that doesn’t seem believable, why be worried about machine learning algorithms running on digital computers instead of matchbox arrays?

View on HN

> If you add more matchboxes to this setup while teaching it to play something else, is it going to become intelligent eventually?

No, because the design uses a different matchbox for every possible state of the game. If you taught it to play chess, a billion matchboxes wouldn't be enough to get four moves in.[1] Yet digital computers can play chess.

On the other hand, if you switched to a completely different design that uses matchboxes to simulate an artificial neural network… a design that would require opening each matchbox on each time step…

…and pumped the number of boxes up to 100 billion (number of neurons in the human brain) or higher…

…and ran the simulation for, perhaps, trillions of time steps (need to simulate evolution, not just the operation of the brain during a single lifetime)…

…and changed the goal from playing a game to something that would require more general logical reasoning...

…then who knows, it might well become intelligent (though it's hard to be sure). But at that point I'd say the lack of believability has more to do with the difficulty of performing 100,000,000,000,000,000,000,000 matchbox operations, than the likelihood of achieving useful results if you did.

[1] http://www.bernmedical.com/blog/how-many-possible-move-combi...

Related: https://xkcd.com/505/.

This comment is addressing weak arguments as if they are strong ones. Of course you are not going to create an artificial general intelligence by taking a machine-learning algorithm and adding more processing power. That doesn't mean there aren't plenty of good reasons to be worried about AGI!

I mean, yes, I realize plenty of people seem to think that you can in fact do that, and it is worth pointing out to these people that they are wrong, but that is not an argument against the more general thesis that there is good reason to be be worried about AGI.

I don't think you've presented a good argument that arbitrarily large amounts of computing power don't result in intelligence.

I think this is a sort of sleight of hand that preys upon people's inability to wrap their heads around large numbers.

I'm reminded of this classic xkcd: https://xkcd.com/505

It feels completely insane that our universe could be simulated by shuffling rocks around. And yet we know, incontrovertibly, that the rocks can do anything a modern CPU can. You just need a lot of rocks -- where "a lot" is not 1 billion, not even close. Scale is a helluva drug.

There is huge gap between special-purpose automatons(which this matchbox array with operator is essentially) and general-purpose computers. You can pretend that turing machine device is a universal computer, but in real world any turing machine would be incapable of doing anything outside of its purpose. The human mind is considered the most "generic" device in the world: the emergence of equally generic AI which can handle any task with human-level genericity and huge computing power will shift the game towards AI dominance. Its like inventing an Oracle that predict the best path to take, it would accumulate influence over our lives even if its just essentially a dumb neural network at its core with zero self-awareness and "sci-fi sentience". Like a 'Paperclip optimizer' gone generic. People seeking Deus ex Machina will consider the advice of a computer more reliable that human advice and place any blame to flaws in algorithms and human programming.Most people can't handle the idea that a program can be wrong by design, not by flawed intent of programming. So such an Oracle will be considered a flawless machine that can't do any wrong, relying on "flawed" human experts diminishes and we begin to live under control of some weird AlphaGo-based paperclip maximizer.

> but in real world any turing machine would be incapable of doing anything outside of its purpose.

If what the brain is doing is computational, then it's not doing anything more powerful than a Turing Machine could do. There would be some Turing Machine design that could do the same processing.

Sounds like the Chinese room thought experiment [1].

[1] https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thou...

The problem with that experiment is that you shouldn’t the concerned with whether the man in the room knows Chinese but whether the room and the man as a system knows Chinese.

Imagine the computer program is a neural network that has been trained with thousand hours of Chinese videos. If the human replicate the computer program including the training phase he may well have learned Chinese at the end

Reply on news.ycombinator.com