To be frank, reasonable people are nervous because muslim ideology, as published, matches the modus operandi of things like ISIS, the states of Pakistan and generally middle eastern countries to some extent.
And frankly what also makes me horribly nervous is that when talking to muslims that are clearly well integrated, at work, or when going out. I can't get them to condemn terrorist behaviour. Not even when we're talking obviously immoral behaviour, like the 1972 olympic attack (just to avoid 9/11), or today ISIS' modus operandi. When it comes to the obvious religious extermination policies that exist in places like Pakistan or Bangladesh, or all over the middle east, again, it does not seem to be possible, in my experience, to hear condemnations of that behaviour. And frankly, when going over the behaviour that muslim history claims "the prophet" had, like exterminating Khaybar, men, women and children over a trivial conflict, again you can't get muslims to condemn that behavior. He, by the way, did not just kill them, but tortured them first, forced kids to watch family members' executions and worse.
They just deny it exists, how it's all a massive conspiracy against islam, muslims, those countries or some other such claim that they can't defend at all. Regularly muslims suggest that the whole of the quran, hadith and other sources has been falsified just for the purpose of that very discussion.
Knowing that these are well-integrated muslims, not "normal" ones is worrying, very, very worrying. To be honest, if someone were to try something like ISIS is doing in Iraq[1] in America, I am absolutely convinced there would not be serious opposition to that from the American muslim community.
I don't get how we can ever really live together with this religion. There simply is no interest in doing that on the muslim side. Yes there is a period of calm that has started with the WWI extermination of the Caliphate by the Turks. I feat that period is ending, and things will get much, much worse in the medium term.
I know I'll get downvoted for this, but I feel this is a legitimate concern.
[1] http://rt.com/news/166436-isis-execution-video-iraq/