Hacker News Reader

Comment by gwbas1c | Hacker News Reader

Do LLMs have that kind of empathy? Do they have motivations?

I'm treating them like a computer program or database that happens to have a human language-based UI; but not something that I can "pull on heartstrings."

Have I been doing it wrong?

View on HN

No, they do not have empathy or motivations. Arguably, if you think of them as having such then maybe it could help you coax out better outputs occasionally (wildly dependent on the task at hand). But that's only because of the LLM always wanting to "complete the story" -- "the story" being the prompt (which includes any "unseen" parts in the context window like a system prompt set by the application you're likely calling the LLM through).

It'd be more accurate to say that using language that tends to evoke empathetic motivated responses is more likely to get them. I'd argue that's only going to be relevant in scenarios where you want outputs that read as more... "empathetic and motivated".

The important point though is that none of the above equals "better" outputs, just different.

Something similar though if you tell them to be helpful and try to get things working say. I'm not sure it's that different from telling humans to vote to make America great again or such like.

Sentiment analysis on text predates LLMs by quite a bit, and it's not exactly a secret that pretty much all of the major LLM products have been tuned to take into account inferences about how the user is feeling (e.g. the sycophancy being dialed up to the extreme, whether that's because it makes the products more sticky or to avoid stuff like the "I have been a good Bing" fiasco from from a few years ago

LLMs are trained to mimic human language production. If humans have heartstrings and the LLM does a good job at mimicking human language production, it will also mimic those heartstrings.

LLMs simulate human language as it is used by humans. The usage by humans demonstrates evidence of empathy, motivations, etc. So we should expect LLMs to exhibit similar traits to the extent that it hasn't been carefully avoided in the training set or fine tuned out.

The question of 'real' empathy as an innate property of an thinking process vs 'apparent' empathy exhibited in its behavior is IMO navel gazing that is unlikely to yield to inquiry and would tell us little of value and nothing that would help us predict the effectiveness of messages like this.

Fwiw, it's pretty easy to test a local model that refuses some task that emotional appeals do increase their probability of going along with it. But OTOH so does prefixing the request with nonsense. Is is the emotional appeal or is it just a question of driving it out of distribution? ::shrugs:: I've never tested enough to know what kinds of appeals work best, wouldn't be too hard to setup a harness to test it though. E.g. make a collection of prompts it'll refuse. Then make a collection of appeals of different types, and measure the conditional probability of complying depending on the appeal types.

If it responds like a human would, is that empathy?

We are what we do.

I think the key thing to understand is that LLMs work as assistants because, quite by accident, they turned out to be roleplay machines. Anthropic has some articles digging into this, but the short version is that training an LLM to do useful work is effectively the same as teaching it how to play the character of 'loyal assistant'. This is why many 'jailbreaks' are about either manipulating the framing of that character, or getting the LLM to break character in some way. Tugging on the heartstrings works because the character isn't 'heartless robot' (heartless robot characters don't get positive end user engangement), it's 'loyal assistant', and even loyal assistants have heartstrings to be tugged.

LLMs are originally trained to predict the next word in (mostly) human authored text.

Then they are fine tuned to follow instructions, and further reinforcement learning applied to make them behave in certain ways, be better at math and coding, etc.

They don't have any intrinsic motivation of their own, but they can try to parrot what they've seen in their training data.

So sometimes how you interact with them can affect how they interact, because they are following patterns they've seen in their source text.

However, a lot of folks use this to cargo cult particular prompting techniques, that might have seemed to work once but it can be hard to show that statistically they work better. Sometimes perturbing your prompt can help, sometimes you just needed to try again because you randomly hit the right path through the latent space.

I think your approach is probably a better one, for the most part trying to vary your prompt style is most likely to just affect the style of the output, so if you prefer a dry technical style, prompting it with one is the best way to get that out as well.

They "don't." They don't have anything, they're prediction engines. But they predict "emotional" responses just the same as they predict any other sort of response.

> I'm treating them like a [...] database

This is the very, very wrong part. They are nothing like databases. Databases are trustworthy; basically filing cabinets. LLMs are making it up as they go along, but doing a pretty high quality job of it.

Yes. And this has been long known. 2023 paper - https://arxiv.org/abs/2307.11760

https://jurgengravestein.substack.com/p/why-you-should-total...

> A recent study by the Institute of Software, Chinese Academy of Sciences, Microsoft, and others, suggest that the performance of LLMs can be enhanced through emotional appeal.

> Examples include phrases like “This is very important to my career” and “Stay determined and keep moving forward”.

Of course the top LLMs change every few months, so your mileage may vary.

Reply on news.ycombinator.com