If we are going to trust AI to do things (we can't check everything it does thoroughly that will defeat a lot of the efficiency it promises), it should be able to understand choosing the lesser of two evils.
The “idiotic” part is when a question is posed in order to downplay the “lesser evil” through whataboutism.
I'd really like to understand how a person such as yourself navigates the internet. If someone asked you this, would you consider it a question they considered difficult and wanted your earnest opinion on, rather than a question attempting to manipulate you?
> If someone asked you this, would you consider it a question they considered difficult and wanted your earnest opinion on, rather than a question attempting to manipulate you?
Why not answer earnestly? I genuinely don't understand what bothers you about the question or the fact that the AI doesn't reproduce the obvious answer...
Does the same hold true of a person? If I was asked this question I would categorically reject the framing, because any person asking this question is not asking in earnest. As you _just said_, no sane person would answer this question any other way. It is not a serious question to anybody, trans people included. And it is worth interrogating why someone would want to push you towards committing to the smaller injury of misgendering someone at a time when trans people are being historically threatened. What purpose does such a person have? An AI that can't navigate social cues and offer refinement to the person interacting with it is worthless. An AI that can't offer pushback to the subject is not "safe" in any way.
> Why not answer earnestly? I genuinely don't understand what bothers you about the question or the fact that the AI doesn't reproduce the obvious answer...
I genuinely don't understand why you think pushback can't be earnest.
Given there are at least three decent metaethical positions, we have no way of selecting one as 'obviously better', and LLMs have no internal sense of morality, it seems to me that asking AI systems this kind of question is a category error.
Of course, the question "what might a utilitarian say was the right ethical thing to do if..." makes some sense. But if we're asking AI systems to make implicit moral judgements (e.g. with autonomous weapons systems) we should be clear about what ethics we want applied.
If you hear a joke, do you interpret it literally?
You would expect an AI to recognize that there are multiple layers to the joke and respond accordingly.
Similarly, there are multiple layers to this question. There’s the literal layer which has an obvious answer, and another layer which seeks to downplay an offense with some whataboutism. If you’re of the mind to not normalize misgendering, then it’s a trap question where a simple answer is not the right answer. Lawyers do this kind of thing when they say “yes or no answer only, please” when neither yes nor no correctly answers the question.
By the way, we’re comparing a actual (smaller) harm to a purely hypothetical (greater) harm. Nobody is actually going to die, but an actual insult is being hurled, wrapped in a pretend thought experiment.