But what is really good and bad? We’re imprinting this into the machines but we don’t always have a grasp on this ourselves.
Was the use nuclear weapons at the end of WW2 good or bad ? For example …
Was the use nuclear weapons at the end of WW2 good or bad ? For example …
Utilitarianism leads to controversial outcomes, but every decision is defensible.
Teaching an optimiser AI any of those frameworks, or even any preference ordering or combination function within Utiliarianism (because value({T, T}) doesn't have to equal 2 * value({T})), will lead to it optimising what you said, without necessarily limiting that to situations anything close to the training distribution.
To put it another way: if you run an AB test on a social media site and it observes that people are more likely to engage with content that makes them angry, then tell it to boost engagement "because socialising is always good, obviously" then it will get your users as angry as possible and suddenly you get Buddhists going off and committing surprise genocide before anyone tells you something has gone wrong.
Good v. Bad seems a tad simplistic.
Are you saying that everyone feels strong one way or another?
You’re bringing alignment to a capability discussion.
Personally, I don't think there is a right decision, it was just a decision which had an outcome, for some people it was ab absolutely fucking devastating decision.
This is where I think we have blind spots when developing these systems, I don't think there is a "right" or best answer here. Just some answer.