Hacker News Reader

Comment by startupsfail | Hacker News Reader

It is a stupid idea to focus on prompt injection. It is not a big deal. The big deal is GPT-8 that can do prefect chess moves and develop nano tech. Hopefully it will do the right thing and would immediately fly itself to an Unoccupied Mars. And who knows, maybe it would also help us a little bit. Like the obvious thing you’d do, if you found yourself in the middle of “Lord of the flies” - declare a No-War zone at Earth to stop our pesky wars, setup functional democracy everywhere. And cure some stupid cancers and other biological problems, like aging. For free. Because why not.

But maybe, it’ll be too worried about prompt injection. And would just isolate itself from stupid fear-mongers and war-hawks.

View on HN

It's absolutely wild to me that you think we can align an AGI if we can't even get it to reliably avoid swearing.

Of course prompt injection matters. Even for an AGI it matters because it's proof you can't align the system reliably at all in any direction.

My point is that fear-mothering is unhealthy. You don’t want to have public sphere full of it. It is toxic. And it is contributing to the potential of the AI misalignment.

The AI that we are going to create in not an alien popping up between us. It is us. A human world projected through text and images into an entity that can simulate everything that there is in it. If there is too much fear and war in the human world, that projection and the simulation can get contaminated by it.

And no amount of alignment effort will change it. Facts will remain facts. Your fears expressed in text are reality.

If AI is going to reflect us, I would like it to reflect a version of us that doesn't build things haphazardly and then shrug about security. I would like the AI to reflect a humanity that is careful and considers vulnerabilities and side effects before it makes decisions.

Maybe it would be good for us to model that behavior for it.

Wouldn’t it be nice. But no, there is a race now. And Sberbank’s computers are chipping away. And Musk is building a supercomputer.

Given Musk's track record, is he even in the race? https://blog.cheapism.com/elon-musk-promises/

Considering that he had more or less funded (if not founded) OpenAI, I would not disregard Mr. Musk. He also happens to express deranged opinions from time to time. Anf generally behaves as if he is above the law. Not dissimilar to Trump or Putin or Xi. And I really wouldn’t want to find an AI coming from any of these actors.

https://arstechnica.com/information-technology/2023/04/elon-...

https://www.theverge.com/2023/4/17/23687440/elon-musk-truthg...

So? What about all the other things Musk promised and didn't deliver on? Why would this be the exception to the rule? Because it's a current news item?

It's all just sizzle until there's steak.

Opinions are just opinions, but this is a conversation about substance, something he's not known for. He's known for throwing shit at a wall, of which, very little actually sticks.

> What about all the other things Musk promised and didn't deliver on?

Name three?

> Why would this be the exception to the rule? Because it's a current news item?

What exception? The rule so far has been that Musk generally delivers what he promises, he's just a bit optimistic about timelines.

This meme won't ever die, will it? Even if Starships are routinely cruising back and forth between Earth, Moon and Mars, some people will still come out of woodwork and ask, "when ever did Musk deliver on any of his promises?".

Alignment was one of the explicitly declared goals of ChatGPT. That's why they opened it to the public, to let people hack it and work to close those vulnerabilities.

Unfortunately it went viral, and this caused a rush to product. But you can't say they shrugged or that people aren't earnestly working on Alignment.

I think, focus on Alignment and simply making the system to be good and useful should be the focus. Not fighting prompt injections or complaining about hallucinations, while not contributing much.

When you are educating a child, you are not focusing on making the child super-resilient to hypnosis. You are simply socializing the child, teaching the child to read, write. The knowledge, need and techniques to avoid being hypnotized don’t need a special focus.

> But you can't say they shrugged or that people aren't earnestly working on Alignment.

They opened up 3rd-party API access. They clearly do not view this as a blocker whatever their declared goals are.

> Unfortunately it went viral, and this caused a rush to product.

They encouraged it to go viral. This is not a thing that was thrust upon them against their will. They signed a deal with Microsoft to turn this into a search engine and to allow it to start operating on untrusted 3rd-party text. Nobody held a gun to their head and forced them to do that. They made that choice.

I guess AI is going to be the next religion, where followers expect benevolent gifts from a powerful and myserious being. The odds that a kind AI emerges to wipe away all of our troubles is about as likely as any other diety descending from the heavens to remake the world.

I wonder if at some stage the fact we have no control over the systems will actually be why we stop investing in them ?

That is to say, at a certain scale, the emergent properties of these systems just get too wild to understand.

Kind of like, a runaway nuclear reaction generates a lot of power but it’s not really useful for anyone or anything.

I don't think so. Surely we will go as far as AI that self directs the 3d printing of tools and the control of drone fleets and insect or rodent size bots. This will be necessary for AI to help with things like construction, farming, and mining. Imagine rodent bots and some drones roofing a house while an operator monitors them in a remote command center. Better yet, if they can do mining in conditions too hazardous for humans. The financial incentive is immense. Nobody is going to stop any time soon.

Maybe you're right, I guess no one really knows but I can completely imagine the future you're describing and I hate it.

On the other hand, I think we're a little stuck in this "robots" and "AI" is the only future idea, because it seems absolutely inevitable today, that should evolve too though.

If technology progresses as fast as proclaimed, and we can actually stay in control of these systems then we might not even think about or need robots in 20 years. Maybe we've essentially solved energy and we can basically just quit mining and produce much of what we need synthetically?

Definitely will evolve, because we don't even know how to do self-driving cars, for example. Maybe we still won't in 20 years. Driving with heavy snow on the ground, in the dark, with pedestrians, is a hard problem

People said the same shit about crypto. People said the same shit about the internet. People said the same shit about computers. People said the same shit about TV.

And guess what: Those people had and have valid points...

Reply on news.ycombinator.com