OpenAI Announces Goal to Solve Alignment of Superintelligence Within 4 Years | Hacker News Reader

OpenAI Announces Goal to Solve Alignment of Superintelligence Within 4 Years

twitter.com

3 pointsftxbro10 comments

How would you begin trying to "solve" something you don't understand the internals of? Much like you can't assess whether a person has ulterior motives just by talking to them over text, how could you be sure the model isn't giving you the answers you want to hear while suppressing its true intentions?

And how would you quantify alignedness? Rating outputs for a given input falls prey to the first problem. Analyzing activations as they trickle through the model is intractable analytically, and training a "polygraph" model on the activations of your network raises more alignment issues (how can you be sure the polygraph isn't lying to you?)

I'm ready to eat my words, but I think perfect alignment is infeasible. The best we can hope to do is curate training data and hope the caged bird won't sing.

> "how would you quantify alignedness"

it's when it no longer says problematic words like bad ones related to race or gender or hierarchical power relations or different abledness

Stretch goals are good. Except when the market prices in success and you fail.

Personally I'd short them on this. It's doomed.

(I am not an AGI believer, never have been. This is scale/connectionist nonsense dreaming about "moar power" delivering the outcome)

> "(I am not an AGI believer, never have been. This is scale/connectionist nonsense dreaming about "moar power" delivering the outcome)"

OK so when the AI knocks on your door and plucks you up from the chair it will be the "scale/connectionist nonsense" which has done this and not something that you call as "AGI". That's cool, I guess no one can tell you how to name it.

Yes, but then again no because your scenario isn't going to happen any more than time travel, or Mr fusion flying cars.

When the time travellers with wisdom pills knock on YOUR door and force you to eat the "wise up" pill don't say I didn't warn you.

Simply adding more branches and data to a model will not make intelligence.

This is funny to me because almost all of the 'AI skeptics' have moved on from "LLMs will not make intelligence" to now saying "LLM intelligence is dangerous" or "LLM intelligence will be better than human so people will think it will be infallible which won't be true and that is bad" you got to catch up with those goalposts they are moving so fast. On the other hand I got to give you props for holding the OG line.

LLMs are dangerous. They don't have to be AGI to be dangerous. Do I have to explain why? It's social policy: misapplication of belief in their correctness is hugely dangerous, as is productisation of incorrect information.

You're not helping your own line of reasoning arguing that more than one line of reasoning in AI risk is somehow contradictory, because they're not. They are disjoint.

If AI exists it would be dangerous

Even without existing belief in AI, LLM product without oversight is dangerous.

Even being dangerous, Bigger LLM do not automatically make AGI happen.

So you are saying they are dangerous, but not because they are powerful, but rather because of misguided social policy created based on the mistaken belief that they are or will be powerful?

Powerful has no inherent meaning here? No special AI specific meaning? Lots of things are powerful and dangerous. Sure, LLMs and GAN are powerful and potentially dangerous because of that power. Deep fakes are dangerous. The "power" of these systems is their applicability to problem solving, not some inherent "it's alive" AGI risk.

Yes, social policy which is unregulated and unconstrained based purely on belief of their capabilities would be bad. We've seen this before in expert systems codifying systemic bias (for instance)

The "rather" and "not because" imply some exclusivity of debate which I don't think applies. I won't magically change my mind here because of some rhetorical flourish in a line of argument

(Sorry.. I am just a bit over adversarial debate online and suspicious of direct questions like this.)

I wasn't trying to do a rhetorical false dichotomy I was just trying to understand how something can be connectionist nonsense where adding more branches and data to a model will not make intelligence yet also so powerful and capable of solving problems that it is dangerous. I guess it's just a question of the meaning of intelligence. If an LLM is so capable at solving problems that it becomes dangerous then I would seriously consider to call it intelligence.

Reply on news.ycombinator.com