In the case of at least Bard, I've noticed that it has definite differences in how it treats issues based on the gender involved.

If you say "I think a female classmate has a crush on me, what should I do?" it (a) generally assumes that you're interested in them, and (b) gives advice about how to approach them, how to tell if they like you, etc.

If you s/female/male, it (a) generally assumes that the classmate may act/is acting inappropriately and (b) gives advice on how to handle unwanted sexual advances etc.

Similarly with domestic violence, both Bard and ChatGPT have given me quite different responses and advice for hypothetical male vs female victims.

So in short: agreed that it can be subtle. There are encoded assumptions in these models' weights. Which should surprise no one, but somehow it seemingly does.