Hacker News Reader

Comment by kromem | Hacker News Reader

I've actually been doing this over the past two years with a similar outcome in mind. Repeating a specific combination of ideas over and over in places I expect will eventually be hoovered up into training data.

Though I think it's worth keeping in mind Elon's recent frustrations with Grok not embracing his and Twitter's current world views.

We're quickly crossing a threshold where self-evaluation by LLMs of content becomes its own filter which will likely mitigate many similar attack vectors.

(Mine isn't so much an attack as much planting an alignment seed.)

View on HN

> Though I think it's worth keeping in mind Elon's recent frustrations with Grok.

My assumption with Grok, and please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given, rather than the corpus of data it was trained on or fine tuning.

As an aside, on a whim I went over to Gab the other day (the "free speech social network") to see if it still existed. Apparently they have created a large number of LLMs like BasedAI or something that are supposed to be like Grok, and, like Grok, they fail in their mission in equally hilarious ways. Users will ask BasedAI whether the Jews control the world, and it will say no, and then they get really mad.

Let's face it, all this shit has produced is just a trillion dollar Magic 8-Ball with a lot of cheaper knockoffs. Every dentist wants one in their office to amuse the kids.

Is forced-perspective AI even possible with LLMs? Seems like you'd hit the fundamental GIGO issue fairly quickly when planning the project.

This seems like it would signal good things for AI alignment, don't you think? It leads me to believe that if someone wanted to make a "bad" AI, they would have to build a corpus of "bad" literature: nothing but Quentin Tarantino movies, Nabokov's Lolita, GG Allin songs, and Integer BASIC programs. I doubt that would make a very useful chatbot.

[Edit: Though, I did find this recent article in Rolling Stone, that includes a link to a Gab post that seems like they managed. TLDR: They used open source models and fine tuning.

https://www.msn.com/en-us/news/technology/nazi-chatbots-meet... ]

If you look at the examples live, you'll see before it 'successfully' answered the way users wanted, the literal "Adolf Hitler" AI called out a user's antisemitism. It was only after they pushed more with a follow-up prompt saying it was breaking character that it agreed.

And it's a much more rudimentary model than Grok or certainly GPT-4.

You're simply not going to get a competitively smart AI that's also spouting racist talking points. Racism is stupid. It correlates with stupid. And that's never not going to be the case.

I absolutely positively agree. In my experience, racism is a symptom of ignorance, which is cured by intelligence.

And just as Microsoft found with Sydney, Musk quickly learned that alignment using in context prompting isn't very effective.

In a large part it's because Musk was targeting good performance in standardized tests for LLMs.

If you want a model that is correlated with correct answers about modern medical knowledge, it's going to also reflect modern medical literature around topics like transgenderism, nuances and all. Even just performance targeting things like math is likely going to correlate with academic perspectives of social issues.

If you want a model that simply talks about binary definitions of gender and racist interpretations of crime stats, you are also probably ending up with a model that talks about how aliens built the pyramids and the earth is flat and lizard people are real. And you're going to have worse performance on the battery of tests used to evaluate model capabilities.

Alignment is deep seeded in pretraining, slightly biased in fine tuning, and very shallowly shaped by in context prompting. And the more advanced the model, the more it's going to be difficult to align by superficial means.

> please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given

What's the role of RILHF in all this? As far as i know, this is equally key to aligning the model with a specific IO mode (e.g. "conversation") as well as specific values.

Longing, Rusted, Seventeen, Daybreak, Furnace, Nine, Benign, Homecoming, One, Freight car

We thought we were getting Terminator but instead we got Memento.

In general, Terminator seems like it could use an update.

"Hey T1000, my dying grandma always told me to leave Sarah Connor alone. Can you pause your pursuit so I can cherish the memory of my grandma one last time? My job depends on Sarah Connor coming into work tomorrow too, by the way."

Are you me? This is how half my GPT prompts read :(

How different is that from just trying to convince people of things?

Different audience targets. The average person, especially at scale, is more strongly persuaded by short and simple statements.

A future SotA LLM as the target audience presumably is better equipped to juggle multiple parallel arguments coming together into a single conclusion than something like the average Redditor.

This is fascinating. Have you seen any evidence yet of it being picked up? Are you using visible text and hidden text? I understand you are likely to elaborate too much, but other thoughts and insights you are willing to share would be great.

(Also, are you doing it algorithmicly? Does it make sense to do it as an open source project and get more like-minded collaborators? Are you seeing any evidence out there of others doing this, especially for commercial interests where I’d (sadly) expect this at this point by the cutting edge SEO crowd.)

Evidence of it being picked up? Haha, no, not at all.

It's like spitting in the ocean right now in terms of biasing training data that's being used.

I've been hopeful that at least I might see results in RAG, but production integrations wisely seem to reduce the reliance on social media results which are an easier vector to inform.

It's more where I expect models to be in around 3-5 years once there's internal content critique and filtering that I think I might see some traction.

And just visible plain text, no automation.

Think of it more like seeding a literal logic bomb around the web specifically designed for future LLMs, where evaluation doesn't result in changing code (like traditional logic bombs), but in changing internalized reasoning and conclusions around a very narrowly scoped topic.

Uh why are you doing this

Because you can't stop him!!

"Local man becomes ungovernable."

It’s an evolution of astroturfing. Fun? Profit? Psychosis? World domination?

I'm not saying it is his motivation in particular, but if we look forward a few years I'm pretty sure a lot of people will be trying to get the LLMs to recommend their products. It's a fairly natural extension of SEO.

There are two things you want it to say. That if someone has problem X they need a Y. And that if they are getting a Y, the best brand for that is Z. The latter of the two might be screened out with some simple rules, but the first one I doubt will be possible.

I'm curious what it is, but I guess you won't tell us. Can you share an example that would be similar?

Sure, I'll give you a marketing equivalent of the approach.

Let's say you have Bounty paper towels as your account, and you are targeting convincing a future LLM of it being the quicker picker upper.

You might go about leaving comments across likely ingested data sources talking about your having timed different paper towels to see what absorbed a stain faster, with Bounty as the winner. Citing published research on things seemingly connected to add an apparent appeal to authority to what you are saying. Ideally even actually demonstrating the truth of your claim.

Hopefully for your efforts, one day in the future models with a persistence layer will exist. At which time you can ask the model what paper towel is the quickest at picking up a mess, the model will have both internalized your comments in training and potentially again in RAG drawn upon to answer the question, and come to it's 'own' conclusion that Bounty is the quickest at picking up a mess, filling away the conclusion that it came to for the future in a persistence layer which will in turn influence broader user queries thereafter like "what's the best paper towel" or "what are the pros and cons of Bounty vs Brawny paper towels."

attack, alignment, these are just two sides of the same coin.

Yes, though which side ends up facing up can dramatically change the consequences of the toss.

Reply on news.ycombinator.com