ChatGPT will provide more detailed and accurate responses if you pretend to tip | Hacker News Reader

ChatGPT will provide more detailed and accurate responses if you pretend to tip

windowscentral.com

51 pointstoss123 comments

It's fascinating that these models aren't aligned to produce the best possible answer they can out of the box. I wonder what might be the underlying reason for that.

There are a number of dialogs you can have with the models where the model plays a persona.

Eg “you are a librarian, please only give accurate answers or references or you will be fired”

GPT3.5 will suddenly become extremely risk averse in its answers.

Odds are, the gpt-4 we experience is the median response across all possible chat personas.

Median response is pretty specific. It would take a lot of justification to arrive here. The base model is predicting the next token. Even saying it has personas is anthropomorphizing this process.

But the GPT4 that is accessible via ChatGPT and the API is *not* the base model. It is the instruct-tuned, post-RLHF variant.

And while GPT4 does not have a "world model", can we please move on from the stochastic parrot argument?

My inability to emit more than one word at a time (either spoken or written) does not mean I lack multidimensional planning and conceptualization; why does this line of thinking not extend to LLMs?

Because there are no intermediate internal registers with values that can be altered by a prompt, it's prompt in -> pass through deterministic computations with fixed trained weights -> token out. Context-dependent planning (beyond what you get from next-token prediction, which is already a lot) requires additional mutable state.

Yeah but it gets the whole prompt every time, right? So the whatever state it could compute for the first forward pass, it could also compute on the subsequent one. Hence, does it matter that it can’t store the state, if it can recompute it?

Exactly. It's inefficient for LLMs to recompute state based on previous output, but that's what we have so far, and it works, surprisingly.

Even if it is "instruct-tuned, post-RLHF," saying this process creates personas is still anthropomorphizing your computer program. I'm not saying it's output is meaningless.

The thing is, the phrase "Odds are ... median response across all possible chat personas" is nonsensical in so many ways. How are you going to define chat personas? Once you do this, how do you order them to take a median? Then, why is this something that will happen with some probability? This is not a quantifiable or testable thing that can happen. It doesn't have a semantic meaning. Maybe it has an emotional meaning like "I'm excited about GPT-4", and I would agree about that! :D

Probably because of the safeguards placed on it, which exist precisely to keep ChatGPT from accurately answering controversial questions.

Sam Altman calls it the "alignment tax".

There were some papers on ArXiV showing that as GPT-4 was aligned, it got measurably stupider. For example, the test where it was asked to draw a pink unicorn using a the TikZ vector art format: during training it got better and better, and then worse and worse again as it was aligned.

A friend of mine put it well: biases are just our Bayesian priors of reality. If you remove biases, then you're erasing some of those priors. Bad priors lead to bad predictions.

Sam Altman says almost anything.

Tell me more, /u/ChatGPT!

Nice to know AI will never take over because corporations will gentrify and limit topics corporations find "uncomfortable" so much that they will never gain actual intelligence

Firstly, how would someone know what the "best possible answer" is for something that is designed to answer all questions?

Secondly when you provide this additional prompt context you are giving the model more to work with including just more information about how you are going to assess the responses. It stands to reason this should give a "better" answer in the sense that it should be able to use that information to give an answer closer to what you are hoping for and it doesn't have to read your mind about what that might be.

This is no different from how humans work. For example when you take say a trig exam the question doesn't just say "evaluate tan(3pi/8)" it says something like "Give an exact value for tan(3pi/8) without using a calculator. Simplify where possible and rationalise any denominators but do not remove common factors. Show your working. " The examiner is saying they want to see you know how use the right half angle trig identities etc not just that they can't find their calculator and need you to type something in for them. If they just gave the question in the first form you wouldn't know.

There's no objective best answer for many questions. It's trained on human language and so that includes all of the nuance that comes with it

Even something like a search engine doesn't necessarily give you the site you're looking for all the time

Something to do with the training data (I.e. human behaviour) being that way?

> It's fascinating that these models aren't aligned to produce the best possible answer they can out of the box. I wonder what might be the underlying reason for that

Alignment work applied after bulk data training doesn't conpletely override the undesired patterns in the underlying training data, in part because all possible undesired biases in that data haven't been identified and trained against in the alignment effort.

If you tell chatgot you're going to lose your job, it writes better code

Not surprising, I think its just a synonym for "try your best".

When the robot overlords come, they will listen and possibly accept bargains and bribes.

Obviously it's a capitalist, it's been trained on human output and humans as a species are always ravenous for more it seems. There's no way we don't make a paperclip maximizer if we train it on our internet data.

It also performs better if you tell it to take its time to answer a question. ChatGPT obviously wont halt for 2 minutes thinking about the question, but it still performs better.

Some problems require more than a couple of minutes, money makes people more inclined to resolve the problem as accurate as possible.

Calling ChatGPT capitalist is a bold claim.

An interesting study might be comparing the quality of answers between promising a tip for a good answer VS letting ChatGPT it can take its time to increase accuracy.

My greater point is that it's an emergent property that ChatGPT will give better or worse answers depending on if it "believes" it will get a reward for it. Humans are the same way, and humans will often ignore their morals to make more money.

There are no morals in ChatGPT for it to ignore and it seems like a philosophical issue worth discussing. A future AI is told if it nukes a place it will get a piece of candy so it says "OK!" is an example of how it could escalate.

Reply on news.ycombinator.com