And we haven't even mentioned income tax. As a US citizen you still have tax liability for income you make in France, meaning you still pay the US a decent amount of taxes you make in France. The US is one of the only countries that does this. So you'll need to hire an accountant familiar with this to do your US taxes, and you still have to pay your French taxes.
And forming a business? Better find someone who knows what to do, and be prepared for it took take a long time, like a year. Small businesses aren't too bad, but if you want employees...that's a totally different story.
Solve the above problems I listed and then France will be attractive to foreigners. Maybe there's a business idea: help companies and people navigate French administration. It's worth a ton of headache.