The sycophancy from Claude is incredibly jarring. I agree with Ethan Mollick that this could turn out to have more of a disastrous impact than AI hallucination.
https://www.linkedin.com/posts/emollick_i-am-starting-to-thi...
https://www.linkedin.com/posts/emollick_i-am-starting-to-thi...
It's really frustrating. I've come to loathe the agreeable tone because every time i see it i remember the times where i've hit this pain point in design.
Not chatty. Unbiased. Avoid use of emoji. Rather than "Let me know if..." style continuations, list a set of prompts to explore further topics. Do not start out with short sentences or smalltalk that does not meaningfully advance the response.
As an aside, I like the further prompt exploration approach.
An example of this from the other day - https://chatgpt.com/share/68767972-91a8-8011-b4b3-72d6545cc5... and https://chatgpt.com/share/6877cbe9-907c-8011-91c2-baa7d06ab4...
One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now).
However, that's a me thing - something that I do (or avoid doing) with how I interact with an LLM. As noted with the stories of people following the advice of an LLM, it isn't something that is universal.
In both of your conversations, there is only one depth of interaction. Is that typical for your conversations? Do you have examples where you iterate?
I think your meta-cognitive take on the model is excellent:
"One part of this in comparison with the linked in post is that I try to avoid delegating choices or judgement to it in the first place. It is an information source and reference librarian (that needs to be double checked - I like that it links its sources now)."
The only thing I would add is that, as a reference librarian, it can surface template decision-making patterns.
But I think it's more like that cognitive trick where you assign outcomes to the sides of a coin, and you flip it, and see how you brain reacts — it's not because you're going to use the coin to make the decision, but you're going to use the coin to induce information from your brain using System 1.
In general, however, I'm looking for the sources and other things to remember the "oh yea, it was HGS-1" that I can then go back and research outside of ChatGPT.
Flipping a coin and then considering how one feels about the outcome and using that to guide the decision is useful. Asking ChatGPT and then accepting its suggestion is problematic.
I believe that there's real damager in ascribing prophecy, decision making, or omniscience to an LLM. (aside: Here's an iterative chat that you can see leading to help picking the right wording for this bit - https://chatgpt.com/share/68794d75-0dd0-8011-9556-9c09acd34b... (first version missed the link))
I can see it as something that's real easy to do. And even back to Eliza and people chatting with that, and I see people trusting the advice as a way of offloading some of their own decision making agency to another thing - ChatGPT as a therapist is something I'd be wary of. Not that it can't make those decisions, but rather that it can't reject the responsibility of making those decisions back to the person asking the question.
To an extent, being familiar with the technology and having the problems of decision fatigue ( https://en.wikipedia.org/wiki/Decision_fatigue ) that, as a programmer, I struggle with in the evening (not wanting to think anymore since I'm all thought out from the day)... ChatGPT would be so easy to let it do its thing and make the decisions for me. "What should I have for dinner?" (Aside: this is why I've got a meal delivery subscription so that I don't have to think about that because otherwise I snack on unhealthy food or skip dinner).
---
One of the things that disappointed me with the Love, Death & Robots adaptation of Zima Blue ( https://youtu.be/0PiT65hmwdQ ) was that it focused on Zima and art and completely dropping the question of memory and its relation to art and humanity (and Carrie). The adaptation focuses on Zima's story arc without going into Carrie's story arc.
For me, the most important part of the story that wasn't in the adaptation follows from the question "Red or white, Carrie?" (It goes on for several pages in a socratic dialog style that would be way too much to copy here - I strongly recommend the story).
Answer concisely when appropriate, more
extensively when necessary. Avoid rhetorical
flourishes, bonhomie, and (above all) cliches.
Take a forward-thinking view. OK to be mildly
positive and encouraging but NEVER sycophantic
or cloying. Above all, NEVER use the phrase
"You're absolutely right."
But for the broader question: I see sycophancy as a double‑edged sword.
• On one side, the Dunning–Kruger effect shows that unwarranted praise can reinforce over‑confidence and bad decisions.
• On the other, chronic imposter syndrome is real—many people underrate their own work and stall out. A bit of positive affect from an LLM can nudge them past that block.
So the issue isn't "praise = bad" but dose and context.
Ideally the model would:
1. mirror the user's confidence level (low → encourage, high → challenge), and
2. surface arguments for and against rather than blanket approval.
That's why I prefer treating politeness/enthusiasm as a tunable parameter—just like temperature or verbosity—rather than something to abolish.
In general, these all-or-nothing, catastrophizing narratives in AI (like in most places) often hide very interesting questions.