zep15

Reasoning means things like this: suppose you are holding a ball and want to make it drop. A rule of inference tells you that if you release it, it will drop. You can then reason out that you should release it.

Sure, the rule of inference may have ultimately been derived from experience, by a process which in some sense involved statistical correlation. But you have to distinguish that ultimate basis for the inference rule from _the process of logical inference itself_. It's the latter that is generally called reasoning.

Reasoning in the above sense is essential to intelligence, even at the toddler level, and the DeepMind work doesn't address reasoning. I think that may be the point the parent was getting at.

Producing a simulated toddler is way beyond our current capabilities, no matter how much simulated experience we give it. We simply don't know (yet) how to program said toddler's brain.

Do many "otherwise smart" people actually believe "superhuman machine intelligence is prima facie ridiculous"? I'd like to see some citations :-). I think smart people tend to have much more nuanced views.

I find it odd that AI risk has become such a hot topic lately. For one, people are getting concerned about SMI at a time when research toward it is totally stalled---and I say that as someone who believes SMI is possible. Stuff like deep learning, as impressive as the demos are, is not an answer to how to get SMI, and I think ML experts would be the first to admit that!

On top of that, nothing about the AI risk dialogue is new. Here's John McCarthy [1] writing in 1969:

> [Creating strong AI by simulating evolution] would seem to be a dangerous procedure, for a program that was intelligent in a way its designer did not understand might get out of control.

Here's someone thinking about AI risk 46 years ago! The ideas put forward recently by Sam Altman and others are ideas that have occurred to many smart people many times, and they haven't really gone anywhere (e.g., at no point between 1969 and now has regulation been enacted). I wish people would ask themselves why that is before making so much noise about the topic. The only people influenced by that noise are laypeople, and the message they're getting is "AI research = reckless", which is a very counterproductive message to be sending.

[1] McCarthy, John, and Patrick Hayes. Some philosophical problems from the standpoint of artificial intelligence. USA: Stanford University, 1968.

I think there's some truth to what you say about Chomskyan linguistics being dogmatic. But linguists aren't really playing the same game as the people in (e.g.) statistical machine translation, so it's hard to say the latter are "winning". Chomskyan linguistics is also just one of the various competing traditions in linguistics.