So they could have easily fabbed something like this, or a tuned architecture specifically designed for the purpose.
Instead, they participate in a program to partner with domestic companies to manufacture their chips: http://trustedfoundryprogram.org/
considering that the entire purpose of NSA in the first place is to provide SIGINT and encrypt or decrypt signals, it's almost a given that they're trying to the best of their ability to crack stuff.
Money buys more commodity hardware faster than the time/money used to develop a chip
It's not hard to make tens, or maybe even hundreds of GPUs beat a specialized chip except for very specific things
And even for something specialized it's probably easier to use an FPGA