Hacker News Reader

Hacker News

Comment by EGreg | Hacker News Reader

Parent Full thread

EGreg

I am shocked that it speaks the way it does when it was trained on random stuff it doesn’t have rights to.

They say they trained it on databases they had bought access to etc. And it seems that way.

Because how does ChatGPT:

1. Do what you ask instead of continuing your instructions?

2. Use such nice and helpful language as opposed to just random average of what people say?

3. And most of all — how does it have a structure where it helpfully restates things, summarizes things, warns you against doing dangerous stuff… no way is it just continuing the most probable random Internet text!!

View on HN

Read the InstructGPT paper and see alpaca. You just need instruction fine-tuning.

Unlike what the other commenters are saying, RLHF, while powerful, isn't the only way to get an LLM to follow instructions.

It is steered by RLHF to give helpful, nice, structured continuations. it was totally trained on random text they never paid a dime for.

Can you please elaborate and answer 1,2,3 directly — I’d love to find out more. Maybw links to the techniques

I am not an expert but as others have said, the InstructGPT and Alpaca models do that

https://arxiv.org/abs/2203.02155#

https://crfm.stanford.edu/2023/03/13/alpaca.html

There is a lot of massaging of inputs and outputs but at the same time: that's done by tweaking the model reinforcing those parts that are desirable and suppressing those parts that are not, not by rewriting the output, though there may be filters that check for 'forbidden fruits'. And it isn't the 'random average' of what people say, that would give you junk, the whole idea is that it tries to get to something better than a random average of what people say.

And by curating your sources you are of course going to help the model to achieve something a bit more sensible as well. Finally: you are probably not looking at just one model, but at a set of models.

I'd like the version without the "sanitized for my protection" part. I cannot imagine being offended by what a computer program generates.

I don't think it is about you per se but about the population in general and I'm quite sure that inside these companies there are unfiltered feeds available. But I can see why in today's sue-happy and sensitive environment a large company would try to not expose itself to that kind of risk. Note that these are exactly the kind of things that the press jumps on and have considerable risk of brand damage.

Oh, I know it's all political. I suspect people pretend to be offended a lot.

https://www.vice.com/en/article/pkadgm/man-dies-by-suicide-a...

Reply on news.ycombinator.com