Hacker News Reader

Comment by slenocchio | Hacker News Reader

I haven't heard a single rebuttal to Eliezer's points that's anywhere near convincing and I would love to hear one. Someone needs to reach out to Sam Altman and persuade him to pause all further development on AI until alignment is figured out. He is the voice that needs to speak on this, he needs to be the adult in the room. For all the idealistic virtue signaling about not taking shares and doing this for the good of the world, it's all bullshit if his actions are indistinguishable from any other predictable self-interested CEO. Tell the world you will stop development until alignment is figured out. That is step 1. Please. GPT 3.5/4 will already bring in billions of dollars for years to come as is.

View on HN

The burden of proof goes the other way. I've never heard a convincing argument that AGI will most likely lead to human extinction. I don't think we have enough information to make strong predictions either way. That doesn't mean we shouldn't put more resources than we currently are into mitigating AI risk. It does mean that we should be highly skeptical of doomsday claims. I posted another comment in this thread explaining why I think his claims aren't justifiable.

Edit: the idea that you can pause AI research until you solve alignment presupposes that alignment is orthogonal to the implementation details of AGI. You need to factor in the risk that pausing will hinder the necessary breakthroughs to actually implement alignment effectively.

Honestly I was really disappointed in his letter. There are no arguments. There are emotions.

AI(LLMs) right now statistically predict what would be next tokens it should put together. The reasoning of ChatGPT which you see, is not what AI thinks, it is what would an average human would think. Because we are predictable. So AI/LLMs put tokens together those are more probable than the others, based on what tokens were in input. It is not even real simulation of an average human. Not intentionally. But we are so predictable that AI simulates us, by putting more probable tokens together one after another... AGI will do the same unless you are intentionally ask it to simulate an intelligent person and set a task to do. This simulated intelligent person may pose a danger, because it will be simulation of a human. Depending on seed it might be simulation of a psychotic mega killer or not. But you get the point, right? The real danger to humanity are the people. Apes with the access to a red button of a doomsday. Thus,by stopping feeding garbage data to AI (like letters from EY, chat and internet crawling datasets, including posts on HN), will effectively make AI less similar to us. It will be not capable of predict and unintentionally simulate an average human. It can be so much better than us, if we feed it with curated data.

  > Tell the world you will stop development until alignment is figured out.

Oh alignment was figured out already, look at who is on the board of OpenAI and the connections they have to US intelligence.

Reply on news.ycombinator.com