Hacker News Reader

Comment by SamBam | Hacker News Reader

> Though I think it's worth keeping in mind Elon's recent frustrations with Grok.

My assumption with Grok, and please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given, rather than the corpus of data it was trained on or fine tuning.

As an aside, on a whim I went over to Gab the other day (the "free speech social network") to see if it still existed. Apparently they have created a large number of LLMs like BasedAI or something that are supposed to be like Grok, and, like Grok, they fail in their mission in equally hilarious ways. Users will ask BasedAI whether the Jews control the world, and it will say no, and then they get really mad.

View on HN

And just as Microsoft found with Sydney, Musk quickly learned that alignment using in context prompting isn't very effective.

In a large part it's because Musk was targeting good performance in standardized tests for LLMs.

If you want a model that is correlated with correct answers about modern medical knowledge, it's going to also reflect modern medical literature around topics like transgenderism, nuances and all. Even just performance targeting things like math is likely going to correlate with academic perspectives of social issues.

If you want a model that simply talks about binary definitions of gender and racist interpretations of crime stats, you are also probably ending up with a model that talks about how aliens built the pyramids and the earth is flat and lizard people are real. And you're going to have worse performance on the battery of tests used to evaluate model capabilities.

Alignment is deep seeded in pretraining, slightly biased in fine tuning, and very shallowly shaped by in context prompting. And the more advanced the model, the more it's going to be difficult to align by superficial means.

This seems like it would signal good things for AI alignment, don't you think? It leads me to believe that if someone wanted to make a "bad" AI, they would have to build a corpus of "bad" literature: nothing but Quentin Tarantino movies, Nabokov's Lolita, GG Allin songs, and Integer BASIC programs. I doubt that would make a very useful chatbot.

[Edit: Though, I did find this recent article in Rolling Stone, that includes a link to a Gab post that seems like they managed. TLDR: They used open source models and fine tuning.

https://www.msn.com/en-us/news/technology/nazi-chatbots-meet... ]

If you look at the examples live, you'll see before it 'successfully' answered the way users wanted, the literal "Adolf Hitler" AI called out a user's antisemitism. It was only after they pushed more with a follow-up prompt saying it was breaking character that it agreed.

And it's a much more rudimentary model than Grok or certainly GPT-4.

You're simply not going to get a competitively smart AI that's also spouting racist talking points. Racism is stupid. It correlates with stupid. And that's never not going to be the case.

I absolutely positively agree. In my experience, racism is a symptom of ignorance, which is cured by intelligence.

Is forced-perspective AI even possible with LLMs? Seems like you'd hit the fundamental GIGO issue fairly quickly when planning the project.

> please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given

What's the role of RILHF in all this? As far as i know, this is equally key to aligning the model with a specific IO mode (e.g. "conversation") as well as specific values.

Let's face it, all this shit has produced is just a trillion dollar Magic 8-Ball with a lot of cheaper knockoffs. Every dentist wants one in their office to amuse the kids.

Reply on news.ycombinator.com