As a non-EU company, you are under no obligation to collect VAT & import duties on shipments to the EU. These will often be collected by shipping agent on behalf of the recipient's tax authority, otherwise it is up to the individual to declare their liability.
To claim this as a reason not to trade internationally is a disservice to your potential customers and to your shareholders, and frankly reads as "we have no idea what we're doing, so we're not going to do it, just in case".
If you're really not sure, pay for some basic tax and legal advice who will tell you there's nothing to worry about. Meanwhile, you're loosing potential revenue and future referrals. (Assuming reasonable standards of product quality...)
If you have any advice on how to ship to EU and pay the taxes on my end (so the customer doesn't pay something separately), please let me know!
It seems like it's possible to prepay VAT if you're shipping from Hong Kong, but not if you're shipping directly from China.
We also currently ship all our shirts to the US for a variety of reasons (including quality control).