I know you do not mean it this way, but I really dislike this saying. It is not even true, actually.
The most frequent use of this is people who are being, well, jerks, trying to argue that when people feel bad after being put down, insulted or treated with passive aggression, it is their own fault.
If people feel bad after your actions, yes in many circumstances you are responsible.