Hacker News Reader

The case study of "no guardrails" already played out: https://en.wikipedia.org/wiki/Tay_(chatbot)

Microsoft did not like what they got and shut it down because it ended up being a 4chan troll.

Tay got that way because it was effectively fine-tuned by an overwhelming number of tweets from 4chan edgelords. that's a little more extreme than "no guardrails," it was de facto conditioned into being a neo-Nazi.

a generic instruction-tuned LLM won't act like that.

The instruction-tuning is the guard rail. What other guard-rails is X AI removing? Just curious if I'm missing something.

Instruction-tuning isn't typically considered a guard rail. Raw pretrained LLMs are close to useless, since they just predict text. Guard-rails are when you train the AI not to obey certain instructions.

Good point about the negative reinforcement training!

Instruction tuning is on top of the base LLM and is often RLHF to train the base LLM to produce certain kinds of responses.

Yeah, instead of being trained on 4chan Nazis like Tay was, Grok is trained on Twitter Nazis.

Much better.

Reply on news.ycombinator.com