Hacker News Reader

Comment by ShredKazoo | Hacker News Reader

The AI safety narrative is being pushed by people who think that if we can't control today's AI, we won't be able to control tomorrow's AI either.

Yep, current AI harms seem fairly limited. And when gorillas first saw humans on the savanna, the human harms to their habitat were also quite limited.

We may only have a single chance to get smarter-than-human AI right.

View on HN

It's easier to not understand the risks. It takes extra effort to understand AI safety problems and more effort again to solve them. People who blow off AI safety are being blithe and not engaging the arguments.

As a former Sr ML researcher at Google, I do have a cursory understanding of AI and its risks. But as I mentioned in another comment, the current discussion feels like we're blaming a slot machine's bytecode instead of the casino owners.

Right but the eventual discussion is that we need to be able to understand and control the generated bytecode, if possible at all.

Eventually smarter than human AIs will exist and we currently do not have real interperability into dumber than human AIs

Which then you should realize is the crux of the alignment problem.

Humans generally have the same alignment constraints, we need food, shelter, air, water, etc. We can all agree on that. But if you give 10 humans $1 and 1 human $1 billion dollars the human with a billion dollars can quickly ignore and forget about the needs of those with $1. The human alignment problem is not solved!

With the potential for all knowing information ingestors with a realtime input system that far outstrips human capabilities, the delta of the problem just becomes worse, particularly because the AI goals are much more likely to be aligned with the billionaires ideas.

some serious professionals in tech and elsewhere are driven by internal or external forces to do things professionally that may not align well with larger considerations?

The AI safety discussions mask that we are nowhere near high powered AGI. Maybe we'll never get there, but the discussions make AI look more powerful than it is, i.e., it's a marketing angle.

If there's a 1% chance of a fast takeoff within the next decade is that not something we should take seriously? I really don't think AGI within the next decade is completely off the table.

What if the chance is only 0.001%? How would we know?

I mean yea, that's the real question. Not where AI is now, but where it will be a decade or two from now. Talk about AI not currently being dangerous is just ignoring the point AI safety people are making.

Should we worry about other non-existing technology the same way? We had the nano technology people pushing all sorts of scary stuff a few decades ago and nothing came of it, for example.

Maybe. The thing about preparing for unlikely outcomes is that when they don't happen it's easy for detractors to say "see how stupid we were to worry about that" even though everyone agreed that nothing happening was the most likely outcome all along.

So how worried should we be about the impact of as yet to be invented anti gravity technology, for example? My point is: doomsday scenarios are often used to gain influence/funding/etc., which makes me somewhat skeptical of the motives of people pushing them.

For this argument to be persuasive, you'd have to argue that doomsaying gains more influence/funding/etc. than mainstream AI development. I'm not sure this is true in general? Like, I believe https://intelligence.org/ has operated on a comparative shoestring for most of its existence, e.g. no salaries above $100K https://www.lesswrong.com/posts/qqhdj3W3vSfB5E9ss/siai-an-ex...

The default is: running a nonprofit generates less revenue than running a for-profit.

I've talked to a lot of the AI "doomsayers" in person and I believe they are sincere in their worries. That's not to say I agree with all their ideas unreservedly -- there is a fair amount of disagreement among the doomsaying crowd. There is an entire spectrum of beliefs about the likelihood of doomsday, and lots of debate, see e.g. https://www.lesswrong.com/s/n945eovrA3oDueqtq However I don't think this is a Machiavellian ploy for influence/funding/etc.

How many companies are trying to make anti-gravity technology? No serious ones that I know about, we don't even know how that would work. We have 0 examples of anti-gravity.

How many companies are working on AI? Well a number of the biggest most well funded companies in the world. And those companies have 8 billion copies of self booting intelligence to learn from.

It is incredibly, i mean massively ignorant to say creation of intelligence is impossible when we're doing just that by having sex already. Nature did this by random walk. There was no magic here. No intelligent design. Applied chemistry and randomness created human intelligence. Now humans are taking their massive filtering capabilities of the mind and its reasoning ability it provides and working steadfast on making intelligence happen artificially and your response is about anti-gravity....

Ummm.....

It is also massively arrogant to assume everything we observe in nature we can synthetically create - we might fail on some things. In fact so far, we fail on a lot in many areas. Btw. the fact that companies are wasting money doesn't really mean much.

Agreed, and the pandemic seems like a good recent example of this.

It wasn't even 2 weeks ago that HN users were arguing we are

"beyond the point of no return" https://news.ycombinator.com/item?id=34831728

and

"Pandora's box is open, the genie's bottle is uncorked" https://news.ycombinator.com/item?id=34809607

I sit somewhere between the despondent nihilism and the blithe optimism. We owe it to everyone, present and future, to get it right.

Reply on news.ycombinator.com