It's poisonous, because you end up blaming yourself because you are not living up to constantly changing external demands.
It's okay to say no to what challenges your well-being. Otherwise you'd be at risk of losing your own autonomy. Accept your limitations, cherish them, and know you don't have to (or even can) change them.
I'm not saying you should say no to everything, but not everything is necessary.