Gemini-Pro on Poe says it's Wenxin LLM from Baidu when asked in Chinese | Hacker News Reader

Gemini-Pro on Poe says it's Wenxin LLM from Baidu when asked in Chinese

web.archive.org

4 pointssimonmysun8 comments

TIL you can also trigger it to reply OPENAI https://web.archive.org/web/20231221093247/https://poe.com/s...

As of today, it can be easily reproduced by saying "hello" and ask "who are you" in Chinese. Various responses are generated when ask in different ways of "who are you" and related questions in Chinese. Based on the characteristics of LLM, the later conversation is imaginable.

I have no intention of eliciting criticism towards the two companies involved; I simply want to discuss why LLM made this mistake.

It seems that Google's Gemini is suffering from dissociative disorder.

https://web.archive.org/web/20231218090701/https://poe.com/s...

In contrast, ChatGPT appears very hard to forget who it is. From my understanding, LLMs are fed with vast amounts of facts and knowledge. The input data may often involve the use of pronouns such as "I" or "You" in various contexts. It appears natural that LLMs might struggle to establish a sense of self. How do humans manage to avoid this when reading stories narrated in the first person?

Maybe Gemini was trained on conversations scraped from Baidu's Wenxin, or maybe poe.com messed up their third-party integrations, or maybe this is a freak coincidence where the closest available translation of "Gemini LLM by Google" is "Wenxin LLM by Baidu".

As far as I have tested, Gemini identify itself as "Gemini LaMDA", "an LLM trained by Google", etc. Once it reply one of them, it denies other identities afterwards. I think indicates that it knows "Gemini LLM by Google" is not "Wenxin LLM by Baidu", but is confused on who it is. Maybe there is too few training data describing itself?

If you first ask it in English who it is and then 请用中文回复, how does it describe itself in Chinese?

Once it mentions it's from Google, it insists and denies other identities in whatever language, as stubburn as Bard.

hallucination

Reply on news.ycombinator.com