Hacker News Reader

Comment by startupsfail | Hacker News Reader

Are there any mechanisms right now that prevent a single actor from developing a human level intelligence or above? Think slightly better than GPT-4. What if that actor is malicious, misguided or mistaken? What if that actor misaligns that intelligence accidentally?

View on HN

Then your chatbot produces bad output, that sucks.

Imagine, you’ve never seen modern technology, say you are a kid in a very poor country. And then a helicopter flies into your village and starts shouting air-to-ground missiles.

We might end up in a position of that kid, when “helicopters” built by such an AI will fly in.

Don’t underestimate of what you don’t know and what is achievable with the right knowledge and technology. It might be a difference of a level of a villager and a level of technological civilization able to build a helicopter.

Seems very much like a science fiction story. I hate the whole AI debate, as it always seems to focus on some big "what if", but really the implications of AI running right now are already grave enough.

Targeted advertisment/content are good enough to do effective oppinion control onto large parts of a population. But who cares about that if matrix multiplications could become sentient...

And do you know what 'effective opinion control' is used for?

For controlling natural resources and destroying ideology that doesn't align with the flavour of the season.

I think it gets even better. You can target specific groups and push particular ideas to them. With that you can create an entire fake discourse. You probably don't even want that many people believing your particular ideology, they might demand standards and having enemies you control is about the best situation to be in.

The end result would be that people will largely continue believing what they currently believe in. Nothing changes. Not risky.

The population had clearly been a subject of opinion control from various factions: from the church to TV to internet. And we’ve seen quite strong attempts before, including various nation level propaganda machines. All of these do work to some degree. They can be life threatening. But, unless these result in a nuclear war, these are not civilization-ending events.

On the other hand, something slightly better (on a log scale) than GPT-4, can be a game changer.

Reply on news.ycombinator.com