It's done the opposite for me. I used to have lower self-esteem. Once I got into shape and visible muscles through shirts, suddenly everyone is nicer to me. Girls are talking to me more. Men are nicer. No bullying.
I quit weed and started lifting weights at the same time. When combined, these two decisions have done wonders for my mental health.