Hacker News Reader

Comment by arresin | Hacker News Reader

> Instagram profile of ‘proud Black queer momma’, created by Meta, said her development team included no Black people

It’s such a weird religion.

Religion?

Isn't the weirdness here that someone at Meta seems to have decided that they should make an AI bot that proudly announced its identity, despite literally not being proud, queer, black, or a mother?

Who does this serve? Why?

I could see a bot that's into fitness or some other hobby having a niche as a wrapper around an AI meant for advice on the hobby, but there's a certain crazy hubris to going "this LLM can really speak as a member of an identity group".

It's performative art. A critique of social constructionism by taking to its extreme.

Our language really needs a new word for what is meant here by "religion".

The word "religion" brings so much baggage with it that it just causes a cascade of communication errors in this context.

Pareto when translated from Italian to English called this "non-logical conduct" and "sentiments" but after a 100 years no one seems to care much about this aspect of Pareto's thoughts.

Sentiments would be a great word for it though instead of "religion".

Woke sentiments instead of "woke religion".

The concept of identity sentiments might even be a little bit of progress.

> Who does this serve? Why?

People type that kind of thing into their insta profile. These AI personas were designed to appear to be using Instagram in the same way people do - which is, typing things like "queer black mom" into their profile

it raises several questions, my first question (besides the obvious question of why is this a thing at all) is does the chatbot actually have the information about the racial background of its dev team or is it hallucinating up an answer when it lacks the data. if it does have that sort of knowledge why personally identifiable information about individual devs accessable to a public facing chatbot and how much of their HR file can be exfitraited via chatbot to anyone that asks it questions, if it doesn't have this information and it just assumes its devs are all white what does that say about the information its trained on

Hallucination. An issue we care about and it just answers regardless of accuracy.

Boo.

So you fact checked this comment before posting, right? Because I am deeply interested if this was hallucination or used RAG and web search to provide the answer, but I can’t tell if you just chose to hallucinate without grounding.

> hallucination or used RAG and web search to provide the answer

Occam's razor - it was an hallucination.

Come on why would anyone put the team's races into RAG or anywhere under the hood lol

Does it seem outlandish that there might have been a press article about the team?

When's the last time you saw an article with entire team responsible with gender and race for a non-flagship product at a company the size of Meta?

Even if the article existed, getting the right article (instead of simply using the first Bing result, which is what the other Meta AI does) and the reasoning in a one shot instant response is better COT models like o1

If you have Messenger, try asking it to list the team behind them or their Tom Brady bot, it will not get far

It's Meta, so this is almost definitely some Llama variant under the hood, especially since these bots have been out for a year, evidently.

It seems these were just insanely poorly trained and gated - getting a bot to comment on its dev team, including in some cases specific names of lead designers, is something that feels crazy to let a bot just do. Even if all the data it would respond with is made up slop, this exact debacle is why you do your best to make a bot not even try to answer this sort of stuff.

As for what information it was trained on that it said its dev team had no black devs, that seems fairly reasonable based on general statistical information and scholarly articles it was trained on. It may still be incorrect, but that's a pretty statistically likely outcome, so it's not surprising an LLM built to parrot the statistically likely outcome would say it.

Reply on news.ycombinator.com