Paying attention to people, listening to them carefully, answering questions and explaining things clearly, talking to them as equals instead of patronizing, etc. is not inherently “manipulative” or “psychopathic”.
The word is just a shorthand for people who are able to express themselves in a way that puts other people at ease and engenders a positive emotional state.
It’s pretty much an unmitigated positive trait. Like any positive trait it can be paired with all sorts of other characteristics positive or negative.
But there’s really just nothing wrong with being charming, and no moral authority that derives from not being so.