Comment by nohat | Hacker News Reader

Comment by nohat | Hacker News Reader

Parent Full thread

Better quality than most such posts, but still seems to be missing the point. The remarkable thing about Bostrom's book is how well it anticipated the objections and responded to them, yet no one seems to bother refuting his analysis, they just repeat the same objections. I actually agree with a decent bit of what he says on these points, though his application of these observations is kinda baffling. He makes a lot of misguided claims and implications about what proponents believe. I'll sloppily summarize some objections to his points.

1. This doesn't really bother making an argument against superhuman intelligence. Yes, of course intelligence has many components (depending on how you measure it), but that's not an argument against superhuman intelligence. I'm reminded of the joke paper claiming machines can never surpass human largeness, because what does largeness even mean? Why it could mean height or weight, a combination of features, or even something more abstract, so how can you possibly say a machine is larger than a human?

2. Mainly arguing about the definition of 'general' without even trying to consider what the actual usage by Bostrom et al is (this was in the introduction or first chapter if I recall correctly). I agree that the different modes of thought that AI will likely make possible will probably be very useful and powerful, but that's an argument for superhuman ai.

3. Well he makes his first real claim, and it's a strong one: "the only way to get a very human-like thought process is to run the computation on very human-like wet tissue." He doesn't really explore this, or address the interesting technical questions about limits of computational strata, algorithm efficiency, human biological limitation, etc.

4. Few if any think intelligence is likely to be unbounded. Why are these arguments always 'x not infinite, therefore x already at the maximum?' He also seems to be creating counter examples to himself here.

5. Lots of strong, completely unbacked claims about impossibilities here. Some number of these may be true, but I doubt we have already extracted anything near the maximum possible inference about the physical world from the available data, which is basically what his claims boil down to.

I haven't read Bostrom's book. I don't think I would enjoy it. Maybe I need to grudgingly read it to be able to respond to what Bostromites say.

Here's the thing. If Bostrom's claims about AI are so strong, why does everyone who's referring to his book as their source of beliefs about the future spout non-sequiturs about AI?

Here's an example. 80000 Hours has a mission that I generally agree with, to find the most important problems in the world and how people can most effectively work on them. But somehow -- unlike cooler-headed organizations like GiveWell -- they've decided that one of the biggest problems, bigger than malaria, bigger than global warming, is "AI risk" (by which they mean the threat of superhuman AGI, not the real but lesser threat that existing AI could make bad judgments). [1]

To illustrate this, they refer to what the wise Professor Bostrom has to say, and then show a video of a current AI playing Space Invaders. "At a super-human level", they say pointedly.

What the hell does Space Invaders have to do with artificial general intelligence?

For that matter, what the hell does deep learning have to do with AGI? It's the current new algorithmic technique, but why does it tell us any more about AGI than the Fourier Transform or the singular value decomposition? I would say this is a bias toward wanting to believe in AGI, and looking for what exists in the present as evidence of it, despite the lack of any actual connection.

Has 80000 Hours been bamboozled into thinking that playing Space Invaders represents intelligence, or are they doing the bamboozling? And if Bostrom is such a great thought leader, why isn't he saying "guys, stop turning my ideas into nonsense"?

[1] https://80000hours.org/career-guide/world-problems/#artifici...

Bostrom is in no way in charge of people who happen to agree with him wrt ai risk. For the book he mostly collected and organized a lot of existing thought on ai risk (not that he hasn't made his own novel contributions). That's very valuable, largely because it makes for a good reference point to contextualize discussion on the topic. Unfortunately the critics don't seem to have read it because (in my experience) they repeat the same objections without reference to the existing responses to those objections.

People do sometimes overblow alphago/ dqn playing Atari, but it's not meaningless. These systems (and other deep learning based systems) can truly learn from scratch on a decent variety of environments. One of the most important unknowns is exactly how difficult various cognitive tasks will prove to be for a machine. Each task accomplished is another data point.

I wouldn't say that DeepMind learns Atari games "from scratch" any more than Deep Blue learned chess from scratch. It learns to play Atari games because it's a machine designed to learn to play Atari games.

I strongly disagree. You don't seem to be aware of the difference in approach between Deep Blue and DeepMind.

Deep Blue was hand-led directly and specifically to solve the problem of chess: It was provided with a library of opening moves, some sophisticated tactical algorithms relevant to the problem of chess, a library of strategies for chess, and so on. Many actual human masters of chess were consulted, directly or indirectly, to help with developing Deep Blue's approach to the problem.

DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way.

It's instructive to note that this is pretty much exactly how, very generally speaking, evolution and intelligence solve the problem of survival: every organism has controls and a glimpse of "game state" and has to learn (collectively as a species, individually as an organism) to play the game successfully.

> DeepMind, on the other hand, was created as a "blank slate" with no more hard-wired instruction than "create optimal algorithms to achieve the winning state, given the inputs." Critically, its learning phase is completely self-directed. Essentially, the box is given access to the controls and the video screen content and then sent on its way.

Have you seen DeepMind algorithm to be able to say this ? Are there other people outside of Google who have seen the algorithm and can confirm Google's press release?

AlphaGo uses TensorFlow.

DeepMind announced the were switching over to use TensorFlow not long ago.

TensorFlow is fully open-source, you can go and read the algorithms on Github. Neural networks aren't particularly hard to understand.

The post you are responding to paraphrases the situation, and neglects to mention goal states/conditions, but is otherwise a fair summary of how neural networks work.

Neural networks, particularly the 'deep' ones, do seem to operate almost like magick. Disbelief is to be expected, but I suggest you take a proper look at them yourself, and see what they can actually do.

If you can grok Python, I highly recommend playing with the Docker image of TensorFlow, which provides a Jupyter notebook (Python in a web interface) where you can easily make use of existing pre-trained models, extend them, and/or build your own models — suitable for application to all kinds of problems / problem domains.

Siraj Raval's videos also cover neural networks in almost every aspect - from coding your own NN from scratch for digit recognition (using just NumPy), through explaining what TensorFlow is, how it works, and how to use it — along with a lot of other machine-learning algorithms, libraries and cloud services.

Many of these topics are just a 5-10 minute video (some with longer live-coding 'deep dive' follow-ups), and many of the code examples are fully functional but just a screen or two of code. https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A

There's also some great new videos on AI / machine learning by Google and Facebook on Youtube that are well worth a watch if the topic interests you.

Maybe you have some legitimate concern about Googles' claim as per their press release and my comment. Who knows, maybe they have some reason to lie about what they did!

But then I wonder why you aren't asking the same question of my parent poster. Has he viewed the DeepMind code, is he qualified to tell us it works the same as chess code? Having made that claim backed on even less evidence than I made mine, I'd say his burden of proof is somewhat greater.

I think there's a heavy dose of press release to what Google is saying. Most people wouldn't call PR puff "lying", but only because standards are low.

I don't think Google has fundamentally different deep-learning technology than everyone else. In fact, TensorFlow indicates that they have the same kind of deep-learning technology as everyone else and they just want to do it more cleanly.

Deep learning is parameter optimization. There are more parameters now, and they optimize more things, but don't get caught up in wild visions of machines designing themselves. Would you consider the bzip2 algorithm to be "self-directed learning"? What's the difference, besides the number of parameters?

The PR people, when they say "blank slate", are discounting all the programming that went into the system because it sounds more impressive that way. This is unfortunate. It has happened in AI for decades. To be a responsible consumer of AI press releases, you need to understand this.

> _I don't think Google has fundamentally different deep-learning technology than everyone else._

That's true, and I never claimed otherwise, but that doesn't help you argue your point - in fact, you just proved yourself wrong. From IBM's press release:

> _Does Deep Blue use artificial intelligence? The short answer is "no." Earlier computer designs that tried to mimic human thinking weren't very good at it. No formula exists for intuition. So Deep Blue's designers have gone "back to the future." Deep Blue relies more on computational power and a simpler search and evaluation function._

I'll summarize for you: Deep Blue and DeepMind, similar names notwithstanding, work in very different ways.

What comparison are you even making here? I know that Deep Blue and Deep Mind are different. There is 25 years (edit: sorry, 20 years) between them! Deep Blue is not deep learning. Did the word "deep", used in two unrelated ways, confuse you?

What I am saying is that I know how deep learning works, actual deep learning of the present, and it does not involve "programming itself".

You are trying to tell me that it must be programming itself, because a press release said so, and press releases would never lie or exaggerate. Based on the current state of AI, this is very improbable. You should focus less on trying to "prove" things with press releases.

I made the comparison to Deep Blue because there is little mystique around it now, and because IBM was even reasonably responsible about avoiding AI hype in their press at the time.

The Atari AI's learning phase is a specifically designed deep neural network. The network did not design itself. It was designed by human programmers.

There are probably numerous variants of it that did not learn to play Atari games, and therefore were not announced.

The human brain didn't design itself either - it's the product of millions of years of evolution via natural selection. But this fact is irrelevant to the topic of whether the approach of DeepMind is fundamentally different from the approach of Deep Blue.

To help you appreciate the difference, try answering this question: Were experts players of Pong, Space Invaders or any other video games consulted to contribute strategies for winning those particular games? Was a redesign required to play a new, different game?

If not, you'll need to tell me where the knowledge of how to win at those games came from. I hope you'll appreciate that the ability to autonomously obtain that knowledge by trial and error rather than design and programming constitutes the fundamental difference.

Any actual chess knowledge in Deep Blue was an optimization. The fundamental algorithm is alpha-beta search, which can learn to play many, many games of strategy. Not Go, as we know now, but the exceptions are few. This is an algorithm that is very adaptable and very good at learning to play games based on a heuristic for whether you are winning.

The DeepMind implementation learns how to play Atari games. You could even say that it's learning how to play one game, where your opponent chooses the game cartridge and your goal is to increment the score counter.

You would absolutely need to redesign DeepMind to play Super Mario Bros.

When you give DeepMind credit for designing itself, you are glossing over the fact that it was programmed by people, most likely for long hours.

Does life have a score? A winning state? Exhaustively defined inputs?

I mean, the way you put it makes it sound like you think it very obviously doesn't. But are you willing to grant that maybe there's some very complicated optimization function for life satisfaction/personal fulfillment/eudamonia/whatever the hell you want to call it? It doesn't have to be practically computable to be interesting, you merely have to be able to do your best to approximate it.

If you deny the existence of any such thing, isn't that tantamount to saying that given the choice between two courses of action, there's never a reason to pick one over the other?

I mean, I feel like as humans, we're trying to follow fuzzy approximations of that function all our lives, whenever we're earning money, or spending time with people we love, or whatever, we're doing it because we think it will bring us happiness or life satisfaction or something.

sometimes there isn't a correct choice... both could be good, both could be bad. The Road Not Taken: http://www.bartleby.com/119/1.html

Absolutely! The goal of the game is to reproduce. To be more specific, it's reproduction at the individual level and survival at the species level.

The genome is a kind of algorithm for hardware to implement a strategy to win at this game in a highly competitive environment.

> I'm reminded of the joke paper claiming machines can never surpass human largeness, because what does largeness even mean?

Link: https://arxiv.org/pdf/1703.10987.pdf

Reply on news.ycombinator.com