> This is why it seems that alignment for a generalized, chat-style AI is a very hard problem, perhaps impossible. You can't align it to solve a certain kind of problem and keep it general to any question. The two goals are in conflict with each other.
I think this problem is going to be solved soon (hopefully), check out for the Persimmon model[0] from Humans&. They train it to mimic humans, it may seems bad but it could be _really_ useful to train an AI to be aligned to humans and understand their goals really well since they can use Persimmon to create a "fake human" following a defined goal that their big AI model will learn to estimate.
It's still early but I think this is what they are heading toward.