Another way to say this is truth matters and should have primacy over e.g. agreeability.

Anthropic used to talk about constitutional AI. Wonder if that work is relevant here.