Hacker News Reader

Comment by burtonator | Hacker News Reader

I am definitely not in favor of OSS license for models >= GPT3...

Not unless they're aligned well.

There are all sorts of horrible use cases that these could be used for.

"Alignment" is just a euphemism for "agrees with me", though. Humans aren't even aligned with each other. Demanding that AI models be "aligned" is essentially a demand that AI only be produced which agrees with your priors.

What your essentially saying is "alignment is very hard", which is what those researching alignment say. And they often use the example of how inter-human alignment is hard as evidence for why it is a hard problem. But saying it is hard is not an argument for why it is essential or not.

While humans are not perfectly aligned, especially if you just look at individuals, we are collectively aligned enough that many people can live together in communities of various scales. That imperfect alignment has been good enough that we have scaled from small tribal groups to an international network of nations. We need AI alignment to be good enough if we hope to continue advancing.

How do you feel about AI which is aligned with Iranian or Saudi cultural norms?

This presumes a lot of breakthroughs in model interpretability, corrigibility and of inner alignment. Since those are a prerequisite for AGI that we can live along side, I'd have some amount of relief that we found at least a temporary solution (but will those solutions scale to ASI?).

Now, if Iran created an AGI that poorly aligned with the global community before other nations had similar AGI, then then I suspect that would result in a future world I wouldn't be happy with. But it could be much better than a world with AGI that is unaligned with any human values, regardless of who created it.

My best case scenario could be AGI being created by a broad international coalition that is able agree with some combination of capabilities and alignment. I'm not very confident that this is our future, though. If anyone is going to do it, I think it is more likely that the USA would be the first to create a culturally aligned AGI. Which of course would still be considered a disaster for incongruent cultures.

The ideal alignment for a computer program is to be aligned completely with the user, prioritizing their needs to the exclusion of all others.

it is true that there are concerns relating to open source and ai, but surely the having them be closed off, manipulated and controlled untrustworthy corporations is worse.

they like the particular techbro that runs the corporation currently. people will find all sorts of ways to lick the boot of private ownership.

Both these options don't feel good to me. Hard to really tell what is ultimately worse, when I can imagine similar outcomes when irresponsible or malicious agents have access to sufficiently powerful AI.

Main positive point for open models is that we will start seeing the abuse sooner and at smaller scales. That might give us more time to build an immune system up against exploits by encouraging us to prioritize development of comprehensive AI safety practices.

For the worst use cases, like scams, why would they respect the license? Though that probably applies for some legal but immoral use cases

There are all sorts of stuff people use www for, but here we are.

Reply on news.ycombinator.com