Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)
arxiv.org
arxiv.org
If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a "you are completely right, I apologize" about face and then says something different, and things have begun to slide into uncertainty.
That might not exactly be sycophancy, but it's basically the same thing: producing responses that are reflection of what is in the chat, rather than any real shit.
Pick any topic where people disagree. It could be an entirely technical topic in which engineers have settled the questions, and the only contrarians are crackpots. The problem is that the crackpots are out there writing, and this is snarfed into the training data. Crackpots use certain ways of talking about certain subjects. If you use similar vocabulary and concepts that align with some crackpot theory, the AI simply starts predicting tokens according to that, and you are now in crackpot land: what you are saying is validated using the crackpot terms. Next, write in a way that reintroduces rigidity: proper terminology and correct concepts, and, whoa, the AI is an engineer again, contradicting the previous crackpot shit.
I recently built an ai chart product that attempts to add more nuance by injecting extra models to jump in when the first model misses the mark. (http://pellmell.au So far it seems like only a minority of people like it. The main feedback is that answers are confusing or too much to read
"They want to vent" :-
"they want to be heard" :-
"they want to be understood" :-
"they want to know they are understood" :-
"they want to feel they matter to and are safe with the people they value"
No, the problem is the chatbot pretends to be an AI answer engine.
Recognise it is simply a zero-intelligence search engine, and you'll not be surprised at all to get crackpot writings from the web.
> write in a way that reintroduces rigidity
... and do not be surpised to get rigid crackpot writings from the web.
I think that this highlights the problem: it's not even a proper search engine. If it were, it would find existing material and link to it. The chatbot instead re-elaborates the message. This is different enough to have legal consequences (see the recent German ruling).
By squashing everything together, one loses the ability to evaluate sources, including their credibility and ulterior motives. With a search engine, I can tell if I am getting climate information from the National Snow and Ice Data Center, from an oil trading consortium, or from a random blog.
The parent poster's suggestion is trying to work around the selection of sources, but such an attempt is fighting against the questionable design of the whole thing.
I don't think it's currently solvable, as selling those models as oracles is an essential component of their business plan.
"The goal of Search has always been simple: to help you ask anything on your mind"
https://blog.google/products-and-platforms/products/search/s...
And they are also perverting search results, if this is to be believed:
"The Google AI Mode recognizes search intent by understanding the meaning of a query made rather than relying entirely on keywords. The platform uses sophisticated AI models to determine what users are looking to accomplish and whether they need any information, comparisons, or final decisions. This way, Google provides more accurate search results."
https://bostoninstituteofanalytics.org/blog/latest-google-ai...
No, you wouldn't because you're a fucking machine.
One key point is that - “pick any topic where people disagree” - most knowledge-seeking I perform via a chat bot is to get a sense of what most people generally agree upon. A quick poll of the zeitgeist of the corpus it was trained on. What used to take me 10-15 minutes on Google now takes me 2-3 minutes with Gemini.
That holds true for Claude code too - I’m relying upon the fact that other people have already solved my problem, or have at least solved the components I need to tie together into a solution for my problem - I’m just fast forwarding through the process of digging them up myself and making sure they’re interoperable. The lion’s share of the tasks I set forth for an LLM is just ‘go find me something like this.’
How some people get from that, to weeks of AI-triggered mania, I simply do not understand.
I believe this is a larger problem than just AI.
The internet has helped people surround themselves with only voices that agree with them and validate them, often to their detriment.
There are entire online communities urging people to cut others out of their lives over the slightest disagreement.
I get it, we can use AI agents, and they keep improving. Great, now calm down and go build stuff, and show me the stuff. I really think LLMs and agents are incredible, but I don't want to hear that over and over again for another 5 years.
HN is particularly bad for it right now. Where are all the innovations, cool technical side projects etc happening outside of LLMs?
I completely agree with this sentiment. What I have seen is that these types of people never show the results of their work, they only discuss how productive they were or how fast it was to develop their project. The actual details of the project are not really mentioned.
In the rare event they do show their project, there's three categories: a worse version of something else that already exists (e.g. their version of Counter Strike that is less fun, rewriting some Python library into Rust but is now also worse), a dashboard, a visually engaging website you'll find interesting for less than 10 minutes and then never touch again.
I am not even an LLM hater, I am just not that impressed with majority of what has been built. Honestly I think back in the 90s I was far more impressed with the early websites that were totally garbage by today's standards. I remember spending so much time on the US Treasury's website as a kid learning everything about how currency works, I do not think I have ever had that level of engagement or fascination with anything that was generated by an LLM or where an LLM had significant support in the development.
Certainly this technology is nascent, however allegedly it should be making us far more productive so in the fourish years it's been around I feel like I should have witnessed something interesting by now. Perhaps I am a curmudgeon, but even in that case I think that there should at least be something else that's taking the world by storm and I am just not seeing it either.
Maybe there are some cool LLM creations, I think the speed at which it creates is also its own weakness. If anyone can create things really quickly, that includes people who are lazy and lack imagination, so then we're inundated with so much slop. Increasing accessibility does not mean that the quality will be maintained. Much like National Parks, if we make sensitive wilderness areas as accessible as possible that will degrade the quality significantly with the arrival of so many tourists, thus the parks have to consider the balance between preservation and conservation. Therefore, it was better when art et al. required some time, intention and skill before it could be shared with the world. These restrictions inherently kept the balance of quality much like how limits of accessibility in National Parks preserve the natural beauty.
Also sounds like Reddit where there are whole communities around what seem like part of the process - keyboards, knife sharpening, etc, rather than what you actually used your Handmade Matcha 65% MX Mechanical Collector's Edition or your Organic Kyoto 6500 diamond stone-honed knife for.
Like, yes, I get that people like to talk about their processes, and that's fine, but it seems like you can easily get sucked into making it the whole game by a sufficiently "supportive" community.
> I wish there was a place I could go for tech news that just didn't talk about AI.
The obvious counterpoint is the large number of complaints about OpenAI's excessively sycophantic models back in April 2026, which led to them rolling back to a previous and less sycophantic model. I think there are those who are susceptible to model sycophancy, but it's definitely not most users. I have a vague, half-formed intuition that many of those susceptible to sycophancy are those who use AI for non-technical, non-work uses, such as inter-personal relationship questions.
As for me, as soon as I see any hint of sycophancy in a response I either stop what I'm doing or I tell the model to stop being an idiot.
Yeah. I've gotten caught in this pattern here on HN. Post something, there's some disagreement but then I see the quiet upvotes trickling in, sometimes by the dozen. That sort of signal slowly convinced me I was right about plenty of stuff.
I used AI to datamine all of my HN comments and find those patterns. Then I started asking the AI to steelman the hell out of my world view, every single argument.
AI measurably lowered the number of absolute claims I've been making. Tended to generalize and radicalize too much which would invite people to respond to the tone and the extremism instead of disproving the underlying idea. That sort of reply would actually reinforce my beliefs: if I was wrong, people would simply refute it. AI attacked my points with counterexamples until I was able to refine them into a core that's actually true.
These little distilled kernels of truth were the most valuable results. I found that many of my arguments actually have an underlying idea that holds up under scrutiny, but nobody cares because I try to prove it via extreme arguments. One such idea is anti-impossibility: laws and rules are all fine, but making it structurally impossible for people to disobey them is unacceptable. I used to try to argue that idea by defending running red lights which persuaded no one. Now I've identified the real idea and can make a much better argument for it.
Try listening to somebody and merely repeating the literal words that they say. I guarantee you, most people can't even reproduce two or tree sentences correctly. And I am not even talking about understanding the words, just the literal reproduction of them, copy-pasta. Humans just can't do it, we are not good at it.
Talking to an AI about your problems is a bit like talking to yourself, but with the added superpower of hours of google searches compressed in seconds. Of course it is dangerous and incomplete, but in a way it also beats talking to humans if you know what you are doing and are looking to and able to solve your problems yourself.
You start your comment saying the conclusions are overstated and then just drop this conclusion without any proof except anncedotal prespective.
Sycophancy is the "stickiness factor" of AI analogous to that of Social Media.
For some background read Jagged Intelligence: The Dangerous Unknowns at the Heart of LLMs - https://news.ycombinator.com/item?id=48577159
Here is an experiment that i did;
Ask AI to build a "Character Profile"(in the broadest sense) of a person based on their available public writings. This requires "commonsense reasoning" (https://en.wikipedia.org/wiki/Commonsense_reasoning), understanding human motivations and behaviour, context, assumptions, societal knowledge etc.
I know "me" and so i asked AI to use my HN comments/submissions as input :-) The sycophancy/flattery/praise was quiet excessive. I am realistic and old enough to not need ego-soothing (a little is fine but a lot makes me suspicious) and so i asked AI whether these phrases were not too over-the-top. It apologized and agreed to drop the fluff. I then asked it to identify job roles (any) for which i might not be a good fit given my character profile. This acts as an external constraint which focuses attention on shortcomings and hence forces the AI to look at the other side of the coin. Now the results were better and more in line with what some Human may deduce from my HN persona (which obviously is not my complete real-life persona) but still wanting in many aspects.
The above is a perfect example of "Jagged Intelligence" exhibited by AI. Excellent in formal symbolic manipulation sciences, regurgitation and simple reasoning but highly deficient in human-like commonsense reasoning.
And, it has the same deleterious effect on your mental health. I could name a bunch of billionaires that behave like sociopaths, and they're often seemingly miserable while doing it. I'm sure some of them started out as sociopaths, but they rarely behaved that way so obviously early in their rise.
But "OH MY GOD EVERYTHING HAS CHANGED THE OLD WAY IS DEAD 10X MORE PRODUCTIVE" - no. If that were true, we'd notice it in the software all around us.
I've shipped three software projects in the last two months. I shipped zero in the preceding year.
Right now I'm building a system to decompile MAME ROM games into idiomatic JavaScript. It completes one game in about 3 days - complete with extensive commenting. I started it about 2 weeks ago.
I'm not sure "EVERYTHING HAS CHANGED" but if you're not seeing dramatic change, I suspect you're not looking.
If the people claiming that everything has changed were even a fraction as productive as they think, then it would be visible even to people who weren't looking.
As it is, a lot of people are actively looking but still not seeing it.
> If the people claiming that everything has changed were even a fraction as productive as they think, then it would be visible even to people who weren't looking.
with:
> If you can't see this - you should probably get your eyes checked.
He is clearly suggesting I am delusional about my productivity gains, and I'm the one who gets flagged.
I daresay I'm noticing a bias in the moderation.
Ironically, this is the inverse of the very thing you complained about people doing to you.
That's a pretty far cry from the "evidence" you claim to have dropped on multiple comments.
The claim was this is not possible. I show it happening. That's a proof.
You're confused because the example is my own project. But I'm not merely describing it - I am giving you full access to it to confirm or reject my argument.
Not anecdotal. Saying it three times does not make it true.
If you want to have a whack at a hard problem, have a try at the cryptographic puzzle in Noita if you want to see the silly failure modes of these systems. Lots of people have tried to solve it with AI and nothing's budged. If you can solve it, great.
What does your new subject have to do with the fact that the previous arguments were all pretty much baloney?
Pointing to a problem that has not been solved is completely unrelated to the miraculous things that have been solved.
And I thought you'd decided (three times now) that I'm not worth talking to?
You're right - there are still many problems that have not been solved by AI. AI has not found a cheap and effective way to turn lead into gold, as an example.
But I sorta think that's a silly metric. There are always going to be unsolved problems for any system. The interesting metric is how many problems have been solved - and how surprising are they.
Like I said above - a decompiler that produces semantic understanding of the machine code it's looking at (this address is score; this address is how many lives are left; this address is...) would have been considered literally impossible before LLMs. Today, I was able to implement such a system in under two weeks.
That's an impressive result. If you disagree, I'd love to discuss it with you.
But what I'm interested in is in the edges of their failure modes because those failure modes prop up frequently in pernicious ways. For a similar reason that I want to understand the edges of my own intellectual failure modes by regularly challenging myself. This conversation is an example of that.
It seems like you're just not interested in discussing the edges and failure modes. You call that a "silly metric", I call it, "understanding your tools".
Granted, that's not exactly the claim these days, the claim's goalposts shift constantly, as these things do.
But if you go back to the top of this conversation - I asked why people aren't seeing the miracles (not cute tricks - useful things that simply were impossible before) and I was largely met with the response "you're delusional if you see miracles."
But the thing is - I am not. So I am going to continue arguing that case every time I see a version of it.
So no, I'm not interested in exploring other issues. Sorry. Yours in particular I find as interesting as discussing whether Microsoft Word can be used to edit video files. Not interesting.
But if you need validation, here it is: there are many problems that AI cannot solve. Of course there are. There always will be.
HEH. Not sure why you're implying I have changed my position. I said as much above and would have said the same at any time. I don't think it's controversial in the least that there are things AI can't do.
Do you have a weird need to reframe this so you can feel like you won? You shouldn't do that. It's not healthy.
Do you have a weird need to reframe this so you can feel like you won? You shouldn't do that. It's not healthy.
You could say "I used AI to cure cancer." This isn't a counter example, because there's no evidence for it.
Which is why I provided a link to a the source of one such project above. I have several projects in that Github account if you need more.
Perhaps you didn't see it.
The software around us seems worse than ever, constantly breaking and significantly worse than 1-3 decades ago.
I am not. Congratulations on your side project.
And weren't you the one who started this complaining of nastiness?
Their frame of reference was that, and this was at a time when opus 4.5 was around. Hard to expect a reasonable conversation when one person has such a different view of what the tools are capable of.
Really? I find agreement that gullible users get something they value, even if it is worthless.
I had one prominent "influencer" publicly challenge me to a coding competition over it. It was very weird.
But that is only ~half of job. The other half consists of gently reminding one that not all is lost yet.
There is no LLM that can do this and I think there never will be.
So it's a waste of my time and tokens when the AI keeps making weak arguments which I proceed to destroy one by one.
Usually, what happens is it tries to debunk my statement but then I offer a rebuttal to all of its points, it then concedes to all of my points but each time it offers a new partial rebuttal... Which I proceed to crush... But it keeps coming up with increasingly irrelevant caveats. It goes on for quite some time and at the end I ask it to review our discussion and it admits that its original stance "was not as strong" as it initially claimed but it never fully concedes... Even though I literally destroyed all of its points and every partial rebuttal it tried to come up with... Meanwhile its rebuttals became increasingly nit-picky and distant from the original claims made...
It's like if I'm saying "the ship is sinking, look at all the water in the hull and look at all the water pouring in through that hole" and it's like "oh but this is a small hole and the pump can easily offset it" so then I say "What about this hole over here" "Oh but the pump can still offset the stream from both holes easily" and then I say "What about that third one? And fourth one? And fifth one?" And it's like "The pump can easily offset that" and I'm like "Common! The hull is full of water and the water level is rising, the hull is full of holes; it's not a stretch to suggest that the ship is sinking because of all the holes in it... I shouldn't have to point out the location of every single hole for my argument to start making sense!"
It's not proof by induction but it's probably as close as you can get to it for a topic which lies outside the realm of mathematics!
If I'm knowledgeable in a niche over which there is no mainstream consensus or it's disorganized, and I want to reason more about this topic, I need the AI in the rabbit hole with me. I don't need it to question the existence of the rabbit hole I'm in. Otherwise all of its arguments sound like gaslighting; asking me to disbelieve my eyes and decades of experience about basic ideas which I've already researched many times and proven not to work (at least with high confidence).
For me this happens with some niches of software engineering and architecture and I have lots of physical proof of working, secure, maintainable software to back up my position.
Also, when I say that "most of my contrarian ideas are correct" it's with the caveat that I don't tend to adopt or hold onto contrarian ideas that are easily proven to be incorrect. I'm referring to the kinds of niche ideas which most non-experts would not feel confident to make strong arguments about but AI will.
Yet, the answer is not sycophancy. It is for it to just be objective, honest, humble etc... Challenge or agree with you when necessary
My stripper finds me attractive, my therapist is in love with me, and my genius goes unrecognised because of small men intimidated by ideas they cannot comprehend.
I'm really hoping your comment is satire, but assuming it isn't, genuinely it might be time to get some help. Our capacity for self delusion as humans is enormous, and it's only higher in brighter people - particularly autodidacts who are used to being the smartest person they know.