I don't think there's an opposition to it. People just don't want to shoot themselves in the foot by gambling on Modafinil having no long term effects.
There is definitely opposition. It's mostly on religious grounds, a la "My beliefs dictate that I should prevent you from 'playing god', because that privelege is reserved for my deit(y|ies).".
Oh, that includes Modafinil, and I'm counting the days until someone notices Phenylethylamine.
You can't ban chocolate in the current world. Dark chocolate can contain up to 0.7% of PEA.