I find Gemini is also hilariously enthusiastic about telling you how amazingly insightful you are being, almost no matter what you say. Doesn't bother me much, I basically just ignore the first paragraph of any reply, but it's kind of funny.
I pushed back and told it it was overreacting and it told me I was completely correct and very insightful and everything was normal with the patient and that they were extremely healthy.
“NTA, the framework you are using is bad and should be ashamed of itself. What you can try to work around the problem is …”
So it seems that LLM "sycophancy" isn't necessarily about dishonest agreement, but possibly about being very polite. Which doesn't need to involve dishonesty. So LLM companies should, in principle, be able to make their models both subjectively "agreeable" and honest.
Recently I tested both Claude and Gemini by discussing data modeling questions with them. After a couple of iterations, I asked each model whether a certain hack/workaround would be possible to make some things easier.
Claude’s response: “This is a great idea!”, followed by instructions on how to do it.
Gemini’s response: “While technically possible, you should never do this”, along with several paragraphs explaining why it’s a bad idea.
In that case, the “truth” was probably somewhere in the middle, neither a great idea nor the end of the world.
But in the end, both models are so easily biased by subtle changes in wording or by what they encounter during web searches among other things, that one definitely can’t rely on them to push back on anything that isn’t completely black and white.