Totally agree. No one buying fitted dress shirts cares if they're $29.99 or $69.99. There is a market that cares, but it isn't affluent geeks reading kickstarter.
People in this domain care about fit, look, quality, American made and then maybe price.
I can fufill all of the requirements above except American Made in SF with my tailor and an $80-90 shirt isn't going to make me cry.
Get the hard stuff done, and then don't worry about the price point so much until you scale. We get it, we'll support you and we'll take the price break later.