Most of the important maths research is funded by the Department of War. NSF is a good chunk, I'd have to look it up, but I wouldn't be concerned about this at all. Math is important for the war planners, always has been always will be.
(I was there when only 40-bit SSL was unrestricted! I very much remember those days! They sucked!)
I guess the hangup I have is that my forays into areas like differential geometry and algebraic topology are extremely abstract, and I cannot control anyone who isn't me who misuses those tools in decidedly not-sober ways. I can only make sure that I don't have a direct hand in that kind of misuse.