Hacker News Reader

Comment by jacquesm | Hacker News Reader

LLMs as a rule seem to be primed to make the user feel especially smart or gifted, even when they are clearly not. ChatGPT is by far the worst offender in this sense but others are definitely not clean.

View on HN

I would pay an extra tiny bit for the LLM to stop telling me how brilliant my idea was when I ask it questions. (Getting complemented on my brilliance is not in any respect indicative of a particular idea being useful, as should be obvious to anyone who uses these tools for more than two minutes. Imagine is a hammer said “great whack!” 60% of the time you hit a nail even if you’re wildly off axis. You’d get a new hammer than would stop commenting, I hope.)

Heck, I can literally prompt Claude to read text and “Do not comment on the text” and it will still insert cute Emoji in the text. All of this is getting old.

>I would pay an extra tiny bit for the LLM to stop telling me how brilliant my idea was when I ask it questions.

gpt-5.2 on xhigh doesn't seem to do this anymore, so it seems you can in fact pay an extra tiny bit

The surprising thing for me was how long it took to get old. I got a reward(and then immediate regret upon reflection) for way too long.

On ChatGPT... Personalization:

    Base style: Efficient
    Characteristics:
      Warm: less
      Enthusiastic: less
      Headers & Lists: default
      Emoji: less

Custom:

    Not chatty.  Unbiased.  Avoid use of emoji.  Rather than "Let me know if..." style continuations, list a set of prompts to explore further topics.  Do not start out with short sentences or smalltalk that does not meaningfully advance the response.  If there is ambiguity that needs to be resolved before an answer can be given, identify that ambiguity before proceeding.

---

I believe the bit in the prompt "[d]o not start out with short sentences or smalltalk that does not meaningfully advance the response." is the key part to not have it start off with such text (scrolling back through my old chats, I can see the "Great question" leads in responses... and that's what prompted me to stop that particular style of response).

John Carmack likes how Grok will tell him he is wrong.

"I appreciate how Grok doesn’t sugar coat corrections" https://x.com/ID_AA_Carmack/status/1985784337816555744

It seems that the main current use of grok is creating nonconsensual sexual images of women and children. I suppose this is going to accelerate the ethics flashpoint a bit.

It’s a free hammer, I’m certainly not stupid enough to pay money for it, I’ll throw it away when I’m done with it or when it stops being free.

Kimi K2 Thinking. Tell it in the system prompt to push back.

They're trained to give responses that get positive ratings from reviewers in post-training. A little flattery probably helps achieve that. Not to mention sycophancy is probably positively correlated with following instructions, the latter usually being an explicit goal of post-training.

I'd be interested to see someone try to untangle the sycophancy/flattery from the modern psych / non-violent communication piece.

In theory (so much as I understand it around NVC) the first is outright manipulative and the second is supposed to be about avoiding misunderstandings, but I do wonder how much the two are actually linked. A lot of NVC writing seems to fall into the grey area of like, here's how to communicate in way that will be least likely to trigger or upset the listener, even when the meat of what is being said is in fact unpleasant or embarrassing or confronting to them. How far do you have to go before the indirection associated with empathy-first communication and the OFNR framework start to just look like LLM ego strokes? Where is the line?

I think the difference between sycophancy and NVC (at least how I learned it) is that a sycophantic person just uncritically agrees with you, but NVC is about how to communicate disagreement, so the other person actually listen to your argument instead of adopting a reflexive defense response.

I think the problem is that telling someone they're wrong without hurting their ego is a very difficult skill to learn. And even if you're really good at it, you'll still often fail because sometimes people just don't want to be disagreed with regardless of how you phrase it. It's far easier for the AI to learn to be a sycophant instead (or on the opposite side of the spectrum, to learn to just not care about hurting people's feelings).

A lot of NVC writing is pretty bad. I recommend going directly to the source https://youtu.be/l7TONauJGfc (3h video, but worth the time)

I think NVC is better understood as a framework to reach deep non-judging empathic understanding than a speech pattern. If you are not really engaging in curious exploration of the other party using the OFNR framework before trying to deliver your own request I don’t think you can really call it NVC. At the very least it will be very hard to get your point across even with OFNR if ot validating the receiver.

Validation being another word needing disambiguation I suppose. I see it as the act of expressing non-judging emphatic understanding. Using the OFNR framework with active listening can be a great approach.

A similar framework is the evaporating clouds of Theory of Constraints: https://en.wikipedia.org/wiki/Evaporating_cloud

Also see Kants categorical imperative: moral actions must be based on principles that respect the dignity and autonomy of all individuals, rather than personal desires or outcomes

> indirection

Isn't nvc often about communicating explicitly instead of implicitly? So frequently it can be the opposite of an indirection.

I guess so? I'm not well-versed, but the basics are usually around observation and validation of feelings, so instead of "you took steps a, b, c, which would normally be the correct course of action, but in this instance (b) caused side-effect (d) which triggered these further issues e and f", it's something more like "I can understand how you were feeling overwhelmed and under pressure and that led you to a, b, c ..."

Maybe this is an unhelpful toy example, but for myself I would be frustrated to be on either side of the second interaction. Like, don't waste everyone's time giving me excuses for my screwup so that my ego is soothed, let's just talk about it plainly, and the faster we can move on to identifying concrete fixes to process or documentation that will prevent this in the future, the better.

As people become more familiar with (and annoyed by) LLMs' tone, I wonder if future RLHF reviewers will stop choosing the sycophantic responses.

You're absolutely right.

Maybe that was necessary to get it passed their CEO...?

I think LLMs reflect the personality of their creators.

LLMs sometimes remind me of american car salesmen. Was the hopeful "anything is possible" mentality of the american dream accidentally baked into the larger models?

Reply on news.ycombinator.com