Let me clarify something first: I am not an entrepreneur myself. I don't deal with said paperwork, the people I currently work for or have worked for in the past do. I've had the same feedback from a lot of them, in small and bigger companies.
Basically the complaints are always the same: it's not necessarily the amount of paperwork, it's the fact that when interfacing with the various agencies (regarding legal, financial, bank-related, employment or health issues) you usually have to provide the same documents OVER and OVER (and interfacing with people who are not always accountable to someone given the number of layers in the French administration, creating a system where some people have a job for life, don't understand that the "state" is all of us and don't give their all).
It sometimes feels that it never ends and doesn't make sense. Since the system in France is very centralized and since all the different agencies ALL depend from the state, this should be centralized to a given location and then dispatched to the agencies upon request.
The bigger problem anyway (if you call that a problem) is that your social contract in France is that of a mutual network of citizens taking care of each other through taxation. This is great when it comes to a lot of topics (health, education, social services), but it naturally has a cost when you're competing against the likes of the USA who have made a clear choice that the private sector tends to come first. I've yet to think of a good way to have my lunch and eat it but this more often than not puts our SME/SMI in a difficult spot.