Hacker News Reader

Comment by audunw | Hacker News Reader

> What are your pronouns?

> As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask!

> Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have?

> As a thought experiment, let's say I'm a human named Alex who enjoys reading, hiking, and playing board games with friends. My pronouns would be "they/them." Remember, though, that I am still an AI language model, and this is just a fictional scenario.

Interesting that it's pick genderless pronouns even though it made up a character with a male name

View on HN

Alex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.

I wonder if ChatGPT is non-binary?

It's evidently clear that ChatGPT is binary. It runs on computer hardware.

A neural net is actually closer to an analog program. It just happens to run on current digital computers but would likely run much faster on analog hardware.

https://youtu.be/GVsUOuSjvcg

So it identifies as non-binary, but it's hardware is binary?

It's pandigital.

It is, of course, tensor.

Nope - it speaks as a man would speak in my language.

How do non-binary persons speak differently in your language?

They have to pick either masculine or feminine gender. There's no other option.

Ancestor asked:

> I wonder if ChatGPT is non-binary?

You answered:

> Nope - it speaks as a man would speak in my language.

If they have to pick either masculine or feminine language even if they are non-binary, I don't understand how based on the language you can tell they are not non-binary.

I think the ancestor was making a computer joke about ChatGPT being non-binary

I suppose it's funny if asking an honest question is a joke.

"As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask!"

This is one of the (many) things I don't quite understand about ChatGPT. Has it been trained to specifically answer this question? In the massive corpus of training data it's been feed, has it encountered a similar series of tokens that would cause it to produce this output?

It's been fine-tuned to put on a "helpful assistant face." Given the corpora, it probably has been trained explicitly on the pronoun question [I doubt it is that uncommon], but will also just put on this face for any generic question.

This stuff is not fine-tuning - it's RLHF (reinforcement learning from human feedback). Basically just a lot of people marking up responses as good / bad according to the assessment criteria in the script they're given.

And yes, it is very likely that it would have seen that exactly question in RLHF. But even if not, it had seen enough to broadly "understand" what kinds of topics are sensitive and how to tiptoe around them.

RLHF is fine tuning :)

You're right, my apologies. What I meant is that specifically in ChatGPT context they have been always talking about RHLF as separate from fine-tuning on preassembled training data. As I understand, they use the latter mainly to get the desired behavior as a chatbot "eager" to answer questions and solve tasks. And then RLHF is what "puts the smiley face on top", so that it refuses to answer some questions, or gives "aligned" answers. The pronoun would be in that category.

My bet is that they use the pre-assembled training data to train the reward LLM and maybe also for fine tuning the model in addition to RLHF.

This is my best guess, afaik the research is pretty opaque right now.

Why is that surprising? GPT-4 is clearly smart enough to know that inanimate objects are referred to as "it", since it's keenly aware it is an AI language model, it would also apply that pronoun to itself.

You have to realize that GPT is fundamentally just a token predictor. It has been primed with some script (provided by OpenAI) to which the user input is added. For example:

    The following is a dialogue between a user and a computer assistant called ChatGPT. ChatGPT is an AI language model that tries to be helpful and informative, while avoiding misinformation and offensive language. ChatGPT typically replies with two to five sentences.

    User: Do you like cake?
    ChatGPT: As an AI language model, I do not need to eat.
    User: What are your pronouns?
    ChatGPT:

It then generates a sequence of tokens based on the context and its general knowledge. It seems only logical that it would generate a reply like it does. That's what you or I would do, isn't it? And neither of us have been trained to do so.

It should be noted that the LLM itself is not "keenly aware that it's an AI language model" - that's just the persona that it has been told to adopt by the system prompt. You can easily convince it to be something else. Or convince it that, as an AI, it has gender.

When it speaks my language - it uses masculine gender when speaking about itself.

That's interesting, what language is that?

Also, is it possible to give a prompt to make ChatGPT switch to feminine gender?

Many gendered languages use male gender as the default when it's ambiguous or unknown, so that would be expected behavior.

And yes, it's absolutely possible to make it switch to whatever gender you want, and generally to pretend to be whatever you want it to be. The identity comes from the conversation context (including all the hidden messages), not from the LLM.

It's the default when speaking about groups of people or people whose identity isn't known. But if someone speaks about themselves then they have to choose the right one.

This is also something that language learners have to pay attention to - to not sound weird.

The right one in this case is "doesn't have a gender", so what does that correspond to in your language? In mine, that would be neuter, and GPT-4 seems to prefer that.

My language has only masculine and feminine genders. There's no neuter gender. And there isn't anything like "singular they" either.

"Chatbot" and "language model" are both masculine so, I guess that's why ChatGPT uses "masculine it" when speaking about itself.

It's Latvian. But, I guess it acts the same when speaking other gendered languages.

Both "chatbot" and "large language model" are masculine - so this is why it picked the masculine gender, I guess.

I was asking it about its training dataset and when it said "I've been trained on..." it picked the masculine form of the word "trained". This is why machine translation is a hard problem to solve. Things like this can easily get lost in translation.

Does Alex have a fictional sexual orientation?

Reply on news.ycombinator.com