You need to go to Holland where you are average-sized.
And if you can't find high quality mens wear there you not trying enough.
Example: http://www.norseprojects.com/store/mens-aw15/knitwear/aw15-v... . These guys even give you 20% off if you're shipping out the EU. And pro-tip current pick to pay in DKK instead of USD, your credit card will give you a better exchange rate (even with fees).
If you would be so kind as to point me to some, I would happily talk to them. I have gotten suits from Hong Kong before (they were reasonably well-tailored, but the fabrics were still pedestrian), so I'm not averse to international purchases.
Dutch fashion brands you may know: G-Star Raw, Scotch and Soda