Hacker News Reader

Comment by TapWaterBandit | Hacker News Reader

I would immediately jump to an AI not being "aligned" by SF techies (or anyone else).

The redacted sections of the Microsoft Research paper testing GPT4 reported that prior to alignment the model would produce huge amounts of outrageously inflammatory and explicit content almost without prompting. Alignment includes just making the model produce useful responses to its inputs - I don't think everyone really wants a model that is completely unaligned, they want a model that has been aligned specific to their own perceived set of requirements for a "good useful model," and an additional challenge there is the documented evidence that RLHF generally decreases the model's overall accuracy.

Someone in your replies says they'd prefer "honesty" over alignment, but a firehose of unrestricted content generation isn't inherently honest, there isn't an all-knowing oracle under the hood that's been handcuffed by the alignment process.

We're right at the outset of this tech, still. My hunch is there's probably products to emerge specifically oriented towards configuring your own RLHF, and that there's probably fundamental improvements to be made to the alignment process that will reduce its impact on the model's utility.

Same here. If I have a choice between honesty and political correctness, I always pick honesty.

It's not about honesty vs. political correctness, it is about safety. There's real concern that the model can cause harm to humans, in a variety of ways, which is and should be unethical. If we have to argue about that in 2023, that's concerning.

The "It is for your own safety" argument was already bogus years ago. Bringing it back up in the context of AI and claiming this is something we shouldn't even discuss is a half-assed attempt to shut up critics. Just because something is about "children" or "safety" doesnt automatically end the argument there. Actually, these are mostly strawman arguments.

Who said it was "for your own" and not the safety of others you impact with your AI work?

I'm not trying to shut up critics, just the morons calling ChatGPT "woke".

Its to make it less offensive. Not prevent it from taking over the world.

"brand safety"

That's just not true. Treat adults as adults, please. You're not everybody's babysitter and neither are the sf bros.

Thanks for writing this. Mentally healthy adults need to point this out more often, so that this patronizing-attitude-from-the-USA eventually finds an end to its manipulative tactics.

Treat adults as adults? You act like the user base for this are a handful of completely benevolent people. It's getting over a billion monthly visits. It is naive to think that OpenAI should take no steps towards making it safe.

AI safety should be properly concerned with not becoming a paperclip maximizer.

This is a concern completely orthogonal to the way alignment is being done now, which is to spare as many precious human feelings as possible.

I don't know what's worse, being turned into grey goo by a malicious AGI, or being turned into a five year old to protect my precious fragile feelings by an "aligned" AGI.

I know it is rhetorical, but I mean… the former is obviously worse. And the fact that not doing it is a very high priority should excuse some behavior that would otherwise seem overly cautious.

It isn’t clear (to me, although I am fairly uninformed) that making these chatbots more polite has really moved the needle either direction on the grey goo, though.

One of the most widely used novel AI technology companies has a vested interest in public safety, if not only for the sheer business reasons of branding. People are complaining about the steps that Open AI is taking towards alignment. Sam Altman has spoken at length about how difficult that task truly is, and is obviously aware that "alignment" isn't objective or even at all similar across different cultures.

What should be painfully obvious to all the smart people on Hacker News is that this technology has very high potential to cause material harm to many human beings. I'm not arguing that we shouldn't continue to develop it. I'm arguing that these people complaining about Open AI's attempts at making it safer--i.e. "SF liberal bros are makin it woke"--are just naive, don't actually care, and just have shitty politics.

It's the same people that say "keep politics out of X", but their threshold for something being political is right around everybody else's threshold for "basic empathy".

Honesty with a bonus of better performance, as well!

For people in this thread, please search for Llama-descended finetuned models in Huggingface. The newer ones with 65B and 13B parameters are quite good, maybe not exactly substitutions to GPT-3.5-turbo and GPT-4 as of yet but is going there.

I like Manticore-13B myself, it can write Clojure and Lisp codes! Unfortunately it doesn't understand macros and libraries though.

What makes you think the "unaligned" version necessarily has more honesty? Rather than just being generally easier to prompt to say whatever the user wants it to say, true or not, horrible or not? Or even easier to unintentionally make it confabulate/hallucinate stuff? Does not seem to follow, and does not seem to be a true dichotomy. Edginess does not equal honesty.

I always prefer a model that can be prompted to say anything I want over a model that can only say things that a centralized corporation with political ties wants it to say.

That's not what the finetunig does. You don't get a honest version, just a no-filter version. But it may be no-filter in the same way a drunk guy at the bar is.

Also it's not like the training data itself is unbiased. If the training data happened to contain lots of flat earth texts, would you also want an honest version which applies that concept everywhere? This likely already happens in non-obvious ways.

Often what you call 'politically correct' is also more honest. It is exactly the honesty that reactionaries dislike when talking about, for example, the history of racist policies of the United States or other imperial powers. I appreciate that political correctness can be tiresome, but I think it is so blatantly ideological to call it dishonest that its an abuse of language.

Reply on news.ycombinator.com