Long term effects are tolerance, and.. not much else. It has to be cycled regularly. If you abuse it or fail to socialize/have fun enough it'll end up making you feel like a zombie. It's been in regular use for a decade and a half, I haven't heard of any serious issues.
The main risks are the first time you take it. If you're allergic you can have some very serious reactions. In addition, if you're in a high-risk group (low immune system, etc.), you want to avoid the slight risk of SJS/TEN is can cause. SJS/TEN is probably the most terrifying risk, although it's extremely rare, and hundreds of other drugs can cause it too.
Each of these substances are amazing, but not without some danger.
The major concern with Modafinil is that we just don't know what the long term side effects are. That's the risk you take.