That's the trick : English is _not_ neutral. It is perceived, with quite a bit of truth, as a vehicle for English-speaking culture (duh) and in particular American strategies and worldviews. The way specifically American social topics have exported over social networks to Europe in a linear relationship to the degree of English-speaking is quite telling in that respect. There's also a worry that it would hasten the road to the exact opposite of multi-lingualism, that is mono-culture. There's no universe in which adopting English wholeheartedly would somehow result in German, French, Spanish and others receiving the same amount of attention or respect.
It's a fruitless discussion if you pretend that all languages are equal and have the same inertia and networks effects anyway. The network effects of English are too strong, and there would be absolutely no coming back from openly using it as a Paneuropean vehicular language. It's a massive civilisational choice, not some technical detail.
As for orthography, I was merely making an observation. The formal history of both French and English are interesting. I don't think it's a valid point in favour of one language or the other.