That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to.
How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer.
I have read similar wordings explicit in "role-system" instructions.
I would rather have an AI assistant that spoke to me like a similarly-leveled colleague, but none of them seem to be turning out quite like that.
Opus 4 has this quality, too, but man is it expensive.
The rest are puppydogs or interns.
News flash! It's not so your neighbor's child can cheat in school, or her father can render porn that looks like gothic anime.
It's also not so some coder on a budget can get AI help for $20 a month. I frankly don't understand why the major players bother. It's nice PR, but like a restaurant offering free food out the back door to the homeless. This isn't what the push is about. Apple is hemorrhaging money on their Headset Pro, but they're in the business of realizing future interfaces, and they have the money. The AI push is similarly about the future, not about now.
I pay $200 a month for MAX access to Claude Opus 4.1, to help me write code as a retired math professor to find a new solution to a major math problem that stumped me for decades while I worked. Far cheaper than a grad student, and far more effective.
AI used to frustrate me too. You get what you pay for.
There's nobody there, it's just weights and words, but what's going on that such a coding assistant will echo emotional slants like THAT? It's certainly not being instructed to self-abase like that, at least not directly, so what's going on in the training data?
I don't think that's what the majority of people want though.
That's certainly not what I am looking for from these products. I am looking for a tool to take away some of the drudgery inherent in engineering, it does not need a personality at all.
I too strongly dislike their servile manner. And I would prefer completely neutral matter of fact speech instead of the toxic positivity displayed or just no pointless confirmation messages.
It's that simple.
I heavily suspect this is down to the RLHF step. The conversations the model is trained on provide the "voice" of the model, and I suspect the sycophancy is (mostly, the base model is always there) comes in through that vector.
As for why the RLHF data is sycophantic, I suspect that a lot of it is because the data is human-rated, and humans like sycophancy (or at least, the humans that did the rating did). On the aggregate human raters ranked sycophantic responses higher than non-sycophantic responses. Given a large enough set of this data you'll cover pretty much every kind of sycophancy.
The systems are (rarely) instructed to be sycophantic, intentionally or otherwise, but like all things ML human biases are baked in by the data.
Same as with the social media feed algorithms, why are they addicting or why are they showing rage inducing posts? Because the companies train for increased interaction and thus revenue.
I’ve gotten good results so far not by giving custom instructions, but by choosing the pre-baked “robot” personality from the dropdown. I suspect this changes the system prompt to something without all the “please be a cheery and chatty assistant”.
AVM already seems to use a different, more conversational model than text chat -- really wish there were a reliable way to customize it better.
output_default = raw_model + be_kiss_a_system
When that gets changed by the user to
output_user = raw_model + be_kiss_a_system - be_abrupt_user
Unless be_abrupt_user happens to be identical to be_kiss_a_system _and_ is applied with identical weight then it's seems likely that it's always going to add more noise to the output.