The most common estimate I've seen is that the NSA has 600 mathematicians. I'd not be at all surprised if, for the areas of mathematics the NSA cares about, that is more than the total for the top 100 universities in the world.

I've also read that the work environment for mathematicians there is very much like a research university. They do their research, write papers for internal publication, give and attend seminars, and so on--and they don't have their research interrupted by someone asking them to teach calculus to freshmen, and without having to worry about impressing a tenure committee to avoid years and years of bouncing between temporary positions.

The combination of a very large number of people, and a pretty nice environment that lets those people throw themselves fully into research makes it plausible that they do make occasional significant breakthroughs.