As an anecdote: I'm 5'6", and I don't think this has been a barrier to anything.

I've had to be more careful about my appearance, of course. My clothes fit well, I'm lean, and I work out.

Despite my diminutive stature, men get out of my way, because I look strong and I walk confidently. Men didn't do this until I got strong.

I get looks from girls all the time. They don't seem to care that I'm in about the 2nd percentile, height wise.

Attitude about height is often more important than actual height for a man. Things would probably be easier if I were taller, but being short is far from the insurmountable obstacle it's made out to be.