If they weren't "terrorists" before, they most likely are now. Think about it, that's the real problem.
You speak as if the rest of the world isn't noticing wtf is happening there, including the families of the detainees. As if these people being kept there wouldn't motivate a lot more people to become terrorists, than freeing them would.
This isn't even burning the village to save the village. This is nuking a whole hemisphere to turn off an alarm clock.