Google actually specifies that they cannot disclose the exact location of one's Google Apps data because it is automatically distributed across three different server farms out of several dozen (or more) potential locations. They also are purposefully vague about disclosing the locations of their data farms, in general. Incidentally, one likely reason for this is security.
I agree that there are some genuine privacy and security concerns about the cloud, but I think most criticisms fall into the unforgivable category of "I don't trust it because I can't see it."
And in this sense, academics refusing to accept the cloud are like academics refusing to accept gravity.