In most cases that setup does not absolve you from tax obligations where you live.
I can see a benefit in setting up a business or branch in EU in general, if you have customers in EU. Otherwise why Estonia specifically?
Because the Estonian government is fairly unique in that it will issue an electronic identification card to anyone who requests it and passes a background check, permitting that person--regardless of nationality--to use most Estonian government services, like business registration and tax filing, from anywhere.
More to the point, though, is that Estonia provides a digital identity and signing system that, among other purposes, allows you to access Estonian government services like forming a company. Since you can form a business under Estonian law, you operate as part of the European Union single market.