Hacker News Reader

Comment by geraltofrivia | Hacker News Reader

I like how they brush off VERY REAL concerns of bias, mis-representation as having "spicy takes" and "a rebellious streak". More power to anyone financing the training of an LLM but if you're too lazy to red-team it, debias it, and expect downwind people to take care of that, say so! Don't pass it off as a unique feature of your LLM.

View on HN

I dunno, I stopped using ChatGPT. The magic’s gone after the fiftieth lecture. I’ll give Grok a shot.

People who don’t want to hear from Grok can just not use it. People who are concerned about what Grok might say to me can rest assured their concern is misplaced.

I may be wrong, and if so, correct me. But do you use ChatGPT as a recreation thing, or as a part of a real world solution? In my current job, we've started offloading a lot of real work to GPT4 (it just works ). Sometimes this is solving some recognition tasks, but in some cases we use it to generate summaries for customers, or interface our helpdesk with the RAG pattern [1].

In this case, bias and unfairness can be a real concern since they will affect the products and the business. Its not about being 'preached to' but about avoiding pissed off customers and decrease of trust from them if our system generates hateful/biased text.

I want to emphasize that it's not about "what Grok might say to me". Of course, not. I've got friends who revel in dark humour and I part-take more than most. But if you've spent millions training an LLM, you're not just going to use it as a toy. It is going to solve some usecase. That usecase will probably affect real people. If your LLM is biased, or has little guardrails, those affects might not all be positive.

[1] https://research.ibm.com/blog/retrieval-augmented-generation...

I think the level of caution should just be a setting.

I dunno, I'll stop eating state of the art fare, the magic has gone. I'll give McDonald's a shot.

People are going to tear thing thing apart and help build a solid case for why these guardrails are built in in the first place.

Elon is ironically going to be a part of the reason that access to foundational models will be banned in the US in the wake of Biden's recent executive order.

I hope that does not happen but I do suspect this will backfire in some way. Hopefully it would ultimately be beneficial, demonstrating why handling these models with care is worthwhile.

That executive order is an affront to intelligence and must be cancelled ASAP by the next president who will hopefully not be a Democrat.

Oh I really hope the next president is a Democrat since we're talking about it! Not a great EO, but Biden has done great so far.

I'm not an American and don't like either of your parties, but just wanted to say I respect the enthusiasm in immediately confronting someone violating social norms. It's people like you that keep communities alive.

I hate to be the one to break it to you, but the GOP doesn't give a rat's ass about you or your rights, either. It's lip service from both parties, both stand to gain from regulatory capture.

And ironically he will be part of the reason to support EU AI act that requires you document and test your foundational models: https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...

Why is that ironic? Musk is one of the most "scared of AI" billionaires you will find.

Is it really ironic if every time he touches AI it ends up causing the opposite of what he tried to do?

"Guardrails"

But yes, you're right. We've seen how dangerous open systems can be. Hackers and scammers have shown us that we need guardrails against operating systems and the web, as you correctly point out. It's time for some legislation that locks the down so the unwashed masses don't have unfettered access to them, don't you agree?

I would much prefer you state directly what you believe than use this mocking facetious tone. It’s not really possible to engage with anything you’re saying because I’d have to guess at how to invert your insinuations.

Anyway I think it’s fine for these systems to be available as open source, I’m not suggesting they be withheld from the public. But when you offer it as a cloud service people associate its output with your brand and I think this could end up harming Twitter’s brand.

The case study of "no guardrails" already played out: https://en.wikipedia.org/wiki/Tay_(chatbot)

Microsoft did not like what they got and shut it down because it ended up being a 4chan troll.

Tay got that way because it was effectively fine-tuned by an overwhelming number of tweets from 4chan edgelords. that's a little more extreme than "no guardrails," it was de facto conditioned into being a neo-Nazi.

a generic instruction-tuned LLM won't act like that.

The instruction-tuning is the guard rail. What other guard-rails is X AI removing? Just curious if I'm missing something.

Instruction-tuning isn't typically considered a guard rail. Raw pretrained LLMs are close to useless, since they just predict text. Guard-rails are when you train the AI not to obey certain instructions.

Good point about the negative reinforcement training!

Instruction tuning is on top of the base LLM and is often RLHF to train the base LLM to produce certain kinds of responses.

Yeah, instead of being trained on 4chan Nazis like Tay was, Grok is trained on Twitter Nazis.

Much better.

By "debias" you obviously mean "bias in the direction of my particular worldview".

Yes. In my case, my particular worldview involves (i) indifferent to gender markers in text (I don't want my LLM to convert me to a female because I'm asking it to write a cover letter for a hairdresser/secretary/nurse role, for e.g.). I don't want my model to write it in AAVE if my name is a common african-american name. I don't want to get the best results only when I write as a middle-aged white man [1]. If I use the model for some decision making, I want it to be fair for subgroups [2], which is a reasonably objective metric [3].

[1] https://aclanthology.org/P19-1339.pdf [2] https://arxiv.org/pdf/1906.09208.pdf [3] https://developers.google.com/machine-learning/glossary/fair...

Will Grok really do any of those things? I would have guessed that RLHF would sort those things out even if it wasn't concerned with debiasing, but just about not making ridiculous mistakes.

These are what debiasing tasks are concerned with, more often than not. RLHF tuning depends greatly on the H part of it, and that data is probably proprietary. So, I guess time will tell. But if I were to hazard a guess based on the content of the announcement , I would say they couldn’t be bothered or couldn’t accomplish proper debiasing/rlhf tuning and therefore worded it so.

Isn't it enough to append "write it as if I'm a middle-aged white man" to your request to obtain the desired output?

No, people like him don't want certain queries to be available to anyone else.

The point I was trying to make was that certain NLP models (maybe these LLMs as well) might give better results if YOU speak to them as a middle aged white man.

To be very clear, I want ALL queries (I presume you mean LLM prompts) to be available to everyone.

Could you also explain what do you mean by people like me? Indians? NLP researchers? People in their thirties? Expatriates?

This just says more about how you think than OP.

Yeah, I'm just not gullible enough to be fooled by vague terms like "fairness" where whoever's in charge is going to decide what is fair and what isn't based on (most likely) some arbitrary worldview (which is most likely woke).

Actually, you just woefully misunderstand that objectivity, while never perfectly attainable, is indeed a metric, and that the bias of a model is inherently linked to the variety and quality of training data.

OP mentioned nothing about fairness; that's orthogonal to objectivity, and you're projecting your worldview onto OP's.

If you go through the links, specifically [3], you will find pretty objective definitions of multiple perspective of fairness. This is a mathematical concept.

In most cases, it's not an "arbitrary world view", it's "reality."

My guess is these hypothetical harms are as imaginary as the supposed collapse of X after letting 80% go.

The "harm" seems to be the public (media) outrage at some inappropriate content produced. Most companies cave immediately at any level of pressure, but I have a feeling Musk won't. He's the type to take it to court if needed.

How do you "debias" an LLM? There is no unbiased standard to test against.

That's not _completely_ true. There are a bunch of datasets (specific to some cultural/lingual contexts) like CrowsPairs[1], StereoSet[2]. There is a lot of work you can do to make sure that the model's predictions are fair as well [3]. But at the end, yes these datasets don't exist at the scale of training sets of these LLMs. Hence red-teaming and RLHF post convergence.

PS: Yes I know CrowsPairs is a dataset with a bunch of flaws. My SO is working, in a team of 10+ linguists and researchers to develop a multi-lingual, generalized version of it which also addresses multiple problems with it. Unpublished work, for now.

[1] https://github.com/nyu-mll/crows-pairs/ [2] https://arxiv.org/abs/2004.09456 [3] https://arxiv.org/pdf/2204.09591.pdf

There are all sorts. You do have to be specific about what you mean by “bias” though. https://arxiv.org/abs/2110.08193 and https://arxiv.org/abs/1804.09301 and https://arxiv.org/abs/2206.04615 …

You can debias it relative to any standard you choose, and maybe should if there is one particularly applicable to the intended use case, but the more important thing for models intended for broad, multi-domain use is to document and disclose biases, and ideally any particularly effective techniques for nudging them for particular applications short of fine-tuning/retraining.

Reply on news.ycombinator.com