Hacker News Reader

Comment by andreyk | Hacker News Reader

See also this summary of the report: https://singularityhub.com/2019/06/02/less-like-us-an-altern...

The argument makes a lot of sense to me as an AI researcher; the idea that we will somehow get to self improving agents that can do any and all things they want to maximize the output of paper clips (as in the famous Bostrom thought experiment) is miles away from how AI is done today in practice AND how software functions. I suspect most software engineers, especially with knowledge of AI, would find the 'AI service' idea which actually reflects how AI is done today much more plausible and worth worrying about than the Bostom science fiction-y fears of AGI...

View on HN

Taking the example of games like chess and Go, where AlphaZero dominates, wouldn't we have to fear people misusing AI by setting up specific arenas and applying AI there long before AI misuses itself?

Yes, the post mentions this:

Drexler is more concerned about potential misuse by human actors – either illegal use by criminals and enemy militaries, or antisocial use to create things like an infinitely-addictive super-Facebook. ... Paul Christiano ... worries that AI services will be naturally better at satisfying objective criteria than at “making the world better” in some vague sense. Tasks like “maximize clicks to this site” or “maximize profits from this corporation” are objective criteria; tasks like “provide real value to users of this site instead of just clickbait” or “have this corporation act in a socially responsible way” are vague. That means AI may asymmetrically empower some of the worst tendencies in our society without giving a corresponding power increase to normal people just trying to live enjoyable lives.

The arenas of finance, business, journalism, and politics already exist, so we should watch for AIs trained to "win" those games.

Exactly ; that's where we area already today in fact - people using facial recognition or DeepFakes in malicious ways.

Reply on news.ycombinator.com