In France (and Germany too apparently), if you are found out - usually by a random tax audit - they will ask you to back-pay employee and employer contributions on the money that you received (so about 60% total)

the issue isn't really the company, as you pointed out, they are not in France, they're not subject to French law. The issue is that as a French tax resident, if you are meeting the criteria for employment (single company representing more than 80% of your revenue, everyday work relationship that is based on subordination etc etc), then you have to pay employment contributions

Why? For the same reason as every other mandatory insurance. If everyone who has a stable, well-paying job is skipping out of employment contributions, it makes the contributions for the others even higher, pushing more people out of the system etc until you have no-one left with employee protections