NSA Collects 'Word for Word' Every Domestic Communication, Says Former Analyst | Hacker News Reader

NSA Collects 'Word for Word' Every Domestic Communication, Says Former Analyst

pbs.org

289 pointschakalakasp106 comments

The "former analyst" in question is Russell Tice, who was terminated by the NSA in 2005 after "publicly urging Congress to pass stronger protections for federal intelligence agency whistleblowers facing retaliation" [1]. There's an interesting exchange from then-director of NSA special access programs Renee Seymour to Tice warning him against testifying on the NSA programs [2] about a half year later.

[1]: http://en.wikipedia.org/wiki/Russ_Tice#Whistleblower [2]: http://www.fas.org/irp/news/2006/04/nsa010906.pdf

That letter seems eminently reasonable to me.

I agree. It's interesting from a historical context.

So if neither intelligence committee (HPSCI, SSCI) has the clearances, who does? How many Congressmen have clearance to be even briefed on programs classified for _both_ military and intelligence 'purposes'?

It surprises me they'd so blatantly seek specific, arbitrary classifications to prevent _the legislators_ from learning the truth.

Nice to see somebody pointing out the obvious bullshit regarding the Utah facility's real purpose (content storage, not metadata).

What kind of iron do you need to run NLP on a day's worth of content (let's limit "content" to voice transcriptions from phone calls)? Suppose I want to pick out "bomb" in near real-time? How many $billions for total information awareness?

Even the best algo is going to produce false positives. At nation scale, that is going to be a massive QA/QC effort.

I'm curious about the storage media they might have... is it just plain old rotational harddrives? Who made them -- Seagate? Western Digital? It's curious that no word has come out from that front -- "wow, this customer is requesting a _LOT_ of storage devices... like, more storage space than what is available in the world".

And really, because of this I think the idea that they might have some in-house innovation is not so far-fetched. We started hearing a lot about inPhase Technologies and such about DVD-sized discs that could store upwards of 6 terabytes back almost 10 years ago, but it amounted mostly to vapourware. We do now have confirmed information from Seagate that they'll start shipping out laptop-sized 2.5" rotational drives that will be able to store around ~60 TB of data within the next 2-3 years. Perhaps NSA has been secretively working with them, if not just producing the devices for themselves?

Forget statements from Seagate or WD. Just look at their bottom line.

The NSA has quite a bit of in-house hardware manufacturing capability, but I don't know if they use it to make storage media.

Uncompressed, 1 hour of phone audio is only ~29MB (8KB/sec*3600s/h). Compressing it for storage can send that way down. Let's assume to 6MB. If every 300M Americans talked for an hour a day, that's only 2TB a day for call audio.

Edit: That's only one-way, so double. But compression can eliminate most of that, as there's usually only audio on one side of a call at a time. Anyways, even if I'm right within a factor of 10 or so, it doesn't really seem like a suspiciously high volume of disks.

(Disclaimer: I'm very drowsy just been woken up due to a datacenter coolant failure so maybe I miscalculated it.)

Wouldn't the information be more useful in text form? Transcripts would take far less space and would probably be necessary for any sort of useful search. They might even call the transcript "meta-data".

Exactly, just speech-to-text the whole thing and you're done. Bonus: it's a lot speedier to search through.

Since you don't have to justify yourself in court, you're never going to have to submit the actual audio to any judge as evidence, so why would you keep it?

Unless the NSA also has speech to text that is decades ahead of everyone else, AND works perfectly well in ~50 languages of interest - because of transcription quality.

Not necessarily - the speech to text just has to be good enough to flag likely use of interesting terms. Those calls could then be fully stored for later analysis by an investigation if it becomes necessary. The vast majority of calls would end up being stored as a text file, but this technique with today's technology would certainly be good enough to flag a reasonably high percentage of calls of interest for audio storage. That phone call from Jimmy to Sally Mae telling her he's going to be late home from work? It doesn't matter if NSA-Siri garbles the translation...

You're off by three orders of magnitude.

6 MB * 300 million = 1716 TB

You're right, but that's only the calls one way. It should really be times by 2.

http://www.wolframalpha.com/input/?i=6+MB+*+300+million

See my comment about two-way. It wouldn't matter much as it's rare that both parties are speaking at the same time.

It's also possible that the compression techniques for long-term storage are vastly superior to realtime codecs. The lowest realtime voice codecs are 300-600 bits per second (they sound like shit), which is 213x compression (so an hour would be under a a meg).

http://www.wolframalpha.com/input/?i=600bps*1h*300000000

81TB a day. Again, this is assuming one hour of calls for 300 million people.

I did a quick search and found this snippet: "A telephia survey said that Americans average 13 talking hours a month – with the 18-24 age group averaging 22 hours."[1]

So that is under half an hour a day average. So, let's assume 300bps (lowest realtime voice codec I'm aware of), half hour a day, I'll stick to 300M people and we get:

http://www.wolframalpha.com/input/?i=300bps*30minutes*300000...

20TB.

So maybe I was only an order of magnitude off. Still pretty sloppy of me.

1: http://www.accuconference.com/blog/Cell-Phone-Statistics.asp...

Actually, it should probably be the logarithm of the population. The calls don't need to be recorded twice.

You're right and that makes far more sense. I apologize for this idiotic calculation. I should have known better because just capturing call signalling on one company's network, was taking 1TB compressed a day. To be fair I did add a disclaimer :\.

The NSA probably purchases hardware from multiple vendors, through multiple fronts.

There are conspiracy theories out there that the whole flooding of the facilities which lead to supply shortage was just a coverup for NSA's order. I don't buy into such FAD, but until a few months ago I wouldn't have bought into the whole NSA thing either. So who knows.

I used to work in Western Digital. This is possibly the easiest conspiracy theory to debunk, since anyone who lived in Bangkok can confirm that the floods were very much real.

The idea was not that they made up the floods, the theory is they made up the supply shortage. They used the floods as a cover up. Now as you said, you worked for WD, and maybe you can prove that their production really did take a huge hit, if yes then well and good, as I said, I am not a fan of the theory either.

Ok, that made slightly more sense. But still, all WD drives are assembled either in Thailand or in Malaysia. Something like 2/3 of all WD drives are made in Thailand. During the floods, production in Thailand was shutdown completely for a month or so. The supply disruption was very much real.

In 1977, NSA were planning to have 157 GB of storage: http://cryptome.org/2013/03/cryptologs/cryptolog_28.pdf

I am sure a lot of it is tape. IBM sells a 900 PB tape library [1]. They could store the actual phone calls on tape, and then only put the necessary metadata for finding it on hard drives.

  [1] http://www-03.ibm.com/systems/storage/tape/ts3500/

That's good PR, too: you can tell the press that "the only thing on our hard drives is metadata" without (technically) lying.

"How much storage" is an interesting question, but it says almost nothing about feasibility. If you have good-enough word-spotting as a front-end to storage, you can target your storage and analysis in a way that takes multiple orders of magnitude off your storage requirements. Then storage becomes like a "surveillance TiVO:" It saves the threads of conversations you want to watch, plus some historical buffer to cover the time it takes to make decisions about what you want to watch.

http://www.governmentcontractswon.com

I might have to scrape this...

Reposting this from a previous thread (https://news.ycombinator.com/item?id=5897654)...

I remember during the Boston bombing investigation, Tim Clemente, a former FBI counter-terrorism agent, told Erin Burnett on CNN that they could go back and get access to the content of the calls between the deceased bomber and his wife, Katherine Russell.

After some Googling, I found a partial transcript of the CNN interview...

"Almost immediately Erin Burnett, the host of CNN's Outfront, wanted to know how the government knew. Aren't phone calls supposed to be private? She interviewed Tim Clemente, a former FBI counter-terrorism agent on May 1, asking:"

  Is there any way … they [the federal investigators] can
  try to get the phone companies to give that up … It’s not
  a voice mail. It's just a conversation. There’s no way
  they can actually find out what [was said on the call],
  right, unless she tells them?

  Clemente:  There is a way. We certainly have ways in
  national security investigations to find out exactly what
  was said in that conversation. It's not necessarily
  something that the FBI is going to want to present in
  court, but it may help lead the investigation … we 
  certainly can find that out.

  Burnett: So they can actually get that? … that is  
  incredible.

  Clemente: Welcome to America. All of that stuff is being 
  captured as we speak, whether we know it or like it, or 
  not.

Source: http://www.thenewamerican.com/usnews/crime/item/15340-boston...

CNN Interview Clip (http://www.youtube.com/watch?v=pPHZrVPt4-U)

CNN Follow-Up Interview (http://www.youtube.com/watch?v=vt9kRLrmrjc)

When I initially saw this on CNN, my first thought was, do they also have access to all photos and videos that are taken and transmitted online?

And if so, couldn't they stitch together a multi-angle montage from all the photos and videos taken at the scene of the Boston bombing (like http://photosynth.net), rather than asking everyone to manually scour through their personal footage?

And ultimately:

Did they want these events to happen?

Fear is the oldest and cheapest manipulation strategy. Religions such as the Catholic Church have successfully used it for thousands of years:

"It's going to be hell, if you don't do as we say."

Perhaps the real leaker here is Tim Clemente, and not Ed Snowden? How many adversaries adapted their communications to avoid the sources and methods Clemente revealed? .. dozens?

Here's a question I've been pondering: Did the NSA want PRISM to leak?

Having everyone in the world know that someone is watching will have significant sociological effects. It is akin to the idea that "God is watching".

Thousands of years ago the fear of God and promise of reward was the carrot and stick that resulted in people working together for the greater good rather than pursuing self interest. It was the invisible hand. Those that united and cooperated survived and evolved, and those that refused to unite died off.

Jonathan Haidt has been talking (http://www.ted.com/speakers/jonathan_haidt.html) and writing (http://www.amazon.com/Jonathan-Haidt/e/B001H6GAXW) about how this was key to and accelerated civilization's evolution -- "it put everyone in the same boat."

This same phenomenon happens at the cellular level -- bacteria group to form mitochondria, and mitochondria to form cells, and so on, each time forming a superorganism (https://en.wikipedia.org/wiki/Self-organization).

E pluribus unum (https://en.wikipedia.org/wiki/E_pluribus_unum) -- "out of many, one."

Ants, wasps, and bees are the canonical example of this, and the beehive is often used to symbolize this concept (https://www.google.com/search?q=beehive+symbolism&newwindow=...). The PRISM name even reflects the symbology -- a honeycomb is made up of prisms (http://en.wikipedia.org/wiki/Honeycomb, http://en.wikipedia.org/wiki/Hexagonal_prismatic_honeycomb).

For the last few hundred years, market forces have been the invible hand (http://en.wikipedia.org/wiki/Invisible_hand), and the corporation emerged as the superorganism. But overtime entropy increases and order erodes. Maybe PRISM is an attempt to reestablish order.

The NSA may see other benefits from PRISM leaking. For example, mining the world's communication data may or may not be practical today, but observing a system changes a system.

How will people's behavior change now that they know they're being watched? It would be easier to detect deltas than to mine the entire system. Turn on the lights. Roaches scatter. Targets reveal.

I've also wondered if perhaps Snowden is still on CIA payroll. Maybe the NSA didn't want this leaked but CIA did.

It's very interesting to think about if Snowden is actually a part of a larger group and is just the front man for leaking what they want out there, regardless of their motivation.

So far he's been presented as a lone wolf with assistance now from groups like Wikileaks but acquiring data on his own and initiating things up himself. But that may not be the case.

They certainly know that sooner or later things risk coming to light.

But it seems more likely that they would have preferred to have this leak and its accompanying chilling effect happen at a later date, when they can be sure that everybody is under their control and their power structure could not be brought down anymore.

> They certainly know that sooner or later things risk coming to light.

General Michael Hayden, former head of NSA, said essentially that on Face the Nation a few weeks ago:

  One of the results of the Snowden leaks is that it
  launched a national debate about the balance between
  privacy and security. 

  I'm convinced the more the American people know exactly 
  what it is we are doing in this balance between privacy 
  and security -- the more they know -- the more comfortable 
  they will feel. 

  So frankly I think we ought to be doing a bit more to 
  explain what it is we're doing, why, and the very tight 
  safeguards under which we are operating.

  ...

  Here's how I do the math. In an ideal world, I would 
  keep all of this secret because any of it that I make 
  public slices some of my operational advantage away from
  me. 

  But here's what I've learned heading up both NSA and CIA.
  You may be able to do one thing one-off based upon narrow
  legalness and the President's authorization, but in
  democracies like ours don't get to do something over a
  long period of time without national consensus. 

  So I'm willing to shave points off of my operational 
  effectiveness in order to make the American people a bit
  more comfortable about what it is we're doing; otherwise, 
  the American people won't let us do it in the first place.

CBS: http://www.youtube.com/watch?v=KpfwjI9Eqy4

"Here's a question I've been pondering: Did the NSA want PRISM to leak?"

Perhaps they have been taking notes from Jeremy Bentham.

"PANOPTICON; OR THE INSPECTION-HOUSE: CONTAINING THE IDEA OF A NEW PRINCIPLE OF CONSTRUCTION APPLICABLE TO ANY SORT OF ESTABLISHMENT, IN WHICH PERSONS OF ANY DESCRIPTION ARE TO BE KEPT UNDER INSPECTION"

...

"To say all in one word, it will be found applicable, I think, without exception, to all establishments whatsoever, in which, within a space not too large to be covered or commanded by buildings, a number of persons are meant to be kept under inspection.

No matter how different, or even opposite the purpose: whether it be that of punishing the incorrigible, guarding the insane, reforming the vicious, confining the suspected, employing the idle, maintaining the helpless, curing the sick, instructing the willing in any branch of industry, or training the rising race in the path of education: in a word, whether it be applied to the purposes of perpetual prisons in the room of death, or prisons for confinement before trial, or penitentiary-houses, or houses of correction, or work-houses, or manufactories, or mad-houses, or hospitals, or schools.

It is obvious that, in all these instances, the more constantly the persons to be inspected are under the eyes of the persons who should inspect them, the more perfectly will the purpose X of the establishment have been attained.

Ideal perfection, if that were the object, would require that each person should actually be in that predicament, during every instant of time.

This being impossible, the next thing to be wished for is, that, at every instant, seeing reason to believe as much, and not being able to satisfy himself to the contrary, he should conceive himself to be so."

http://cartome.org/panopticon2.htm

Really thought provoking post, thanks.

How will people's behavior change now that they know they're being watched?

Here's one possibility.

The really dangerous people - those few actual, professional-grade, funded and organized terrorists - may contract black-market hackers to flood the internet and cell networks with red-herring terror-signature traffic, rendering the entire surveillance system useless for its original purpose. For just a tiny fraction of its cost.

In order to justify the construction and continued funding of the system, its mission will have to creep, and we'll all soon have the police knocking on our doors because of our internet searches.

There is a difference between self-organization and falling prey to parasites. It's the difference between everybody being in the same boat, and almost everybody being in the same pot.

http://en.wikipedia.org/wiki/Eusociality

^ Not what we're dealing with here. Nowhere near.

It wouldn't even have to be technically true. If enough anecdotal evidence "leaked" you could get a lot of people to accept the idea and behave accordingly. It's also a classic move by tyrannical authority.... get the people to believe that the authority is omnipotent and resistance is futile.

If this was their strategy, it would be an extremely risky one from a political point of view.

The NSA's endless budget expansion is now at risk, while almost half of the House of Reps voted to nearly shut the program down last week.

No doubt, the next mid-term cycle will be dominated by people on the left and right, campaigning on the promise to shut down these insane programs. The next campaign for President will likely turn out the same.

From Obama's perspective, this is a complete nightmare. His legacy as President will be tarnished, with even the true believers now deeply disillusioned.

I'm sorry to say that pretty much all the biological examples and analogies you give are either wrong or at best controversial and not widely accepted. Bacteria do not group to form mitochondria; mitochondria do not group to form cells. Mitochondra don't even cooperate "willingly" with their host cells, but rather the host cell must incentivize them by withholding essential nutrients from the "slackers" until they shape up. Insect colonies are not "superorganisms"; the individuals within a colony act selfishly, doing things like secretly laying eggs to propagate their genes and sometimes even trying to "usurp the throne" by killing and replacing the queen. The concept of a "superogranism" as a group of individual organisms that evolve together is not generally accepted by biologists [1]. I'm not familiar with Jonathan Haidt's work, so I can't comment directly on that, but your paraphrase above (e.g. "Those that united ... evolved") sounds suspiciously like group selection, and must answer to the same criticisms [1].

The concepts of entropy and order don't really apply in a social system. In many industries the only stable equilibrium is a monopoly [2], which is arguably the least entropic "market state" possible. The "optimal" market according to capitalism is perfect competition among a large number of small independent firms, which is arguably the maximum-entropy state. So it seems that increased entropy should be a good thing.

The word "prism" has a mathematical meaning and a physical meaning, and I think it's pretty clear that when the NSA called their spying program PRISM, they were invoking the physical meaning of a tool that separates light into its components rather than the mathematical meaning of a solid formed by two identical polygons whose sides are connected by parallelograms.

This might seem like nitpicking. If it was just a few examples in your post that were slightly inaccurate, I wouldn't have cared. But your post is clearly trying to give a bunch of examples in order to establish a pattern, and when nearly all of your examples are wrong in significant ways or based on controversial theories, I feel it undermines your the point you were trying to make.

Finally, suggesting that the purpose of a vast surveillance program is anything other than surveillance is going against Occam's razor, and doesn't seem to make sense to me.

[1] http://en.wikipedia.org/wiki/Group_selection#Criticism

[2] http://en.wikipedia.org/wiki/Natural_monopoly

> I'm sorry to say that pretty much all the biological examples and analogies you give are either wrong or at best controversial and not widely accepted.

Many in today's foreign policy circles agree with Haidt's analogies -- see Foreign Policy's "Top 100 Global Thinkers in 2012" (http://www.foreignpolicy.com/2012globalthinkers?wp_login_red...).

Controversial or not, if policy makers hold the idea, it may be sufficient to justify/sell the strategy. And since when has universal buy-in been prerequisite to a course of action?

> Bacteria do not group to form mitochondria;

"The Evolution of the Cell" (http://learn.genetics.utah.edu/content/begin/cells/organelle...)

"Cells within cells: An extraordinary claim with extraordinary evidence" (http://undsci.berkeley.edu/lessons/pdfs/endosymbiosis.pdf)

Mitochondrion Origin (http://en.wikipedia.org/wiki/Mitochondrion#Origin)

Endosymbiotic Theory (https://en.wikipedia.org/wiki/Endosymbiotic_theory)

> mitochondria do not group to form cells.

Eukaroyte (http://en.wikipedia.org/wiki/Eukaryote) vs. Prokaryote (http://en.wikipedia.org/wiki/Prokaryote)

> Mitochondra don't even cooperate "willingly" with their host cells, but rather the host cell must incentivize them by withholding essential nutrients from the "slackers" until they shape up.

Haidt uses the term "freeloaders" instead of "slackers", but incentivizing the freeloaders to "shape up" is exactly the point of the analogy.

> I'm not familiar with Jonathan Haidt's work, so I can't comment directly on that

All of his writings and talks touch on it, but here's a good overview:

"Jonathan Haidt: Religion, evolution, and the ecstasy of self-transcendence" (http://www.ted.com/talks/jonathan_haidt_humanity_s_stairway_...)

Papers & Citatations (http://scholar.google.com/citations?user=VafYYacAAAAJ&hl=en)

Google Tech Talk: "Hive Psychology and the Moral Life of Organizations" (http://www.youtube.com/watch?v=2APK3tlPL_0)

> The concepts of entropy and order don't really apply in a social system.

http://en.wikipedia.org/wiki/Social_entropy

https://en.wikipedia.org/wiki/Economic_entropy

https://en.wikipedia.org/wiki/Entropy_and_life

https://en.wikipedia.org/wiki/Entropy_(disambiguation)

> In many industries the only stable equilibrium is a monopoly [2], which is arguably the least entropic "market state" possible. The "optimal" market according to capitalism is perfect competition among a large number of small independent firms, which is arguably the maximum-entropy state. So it seems that increased entropy should be a good thing.

It's about balance. Unrestrained corporate growth in its current form is unsustainable to the biosphere. We have limited natural resources. What's good for one group may be detrimental to the whole.

> The word "prism" has a mathematical meaning and a physical meaning, and I think it's pretty clear that when the NSA called their spying program PRISM, they were invoking the physical meaning of a tool.

I don't know how you can argue anything the NSA does is "pretty clear." Look up doctrinal meaning vs literal meaning, or double entendre (http://en.wikipedia.org/wiki/Double_entendre).

I want to see and hear more from Clemente

So here's my startup idea:

Stenographically-encoded messages in high-def porn. It's secure communications with a built-in subscription model!

Porn accounts for a huge chunk of internet traffic today, and video doesn't compress too well. Get enough people exchanging messages in porn videos and all of a sudden, those listening in have a whole hell of a lot more bytes to sift through.

(I'm only half joking.)

Real terrorists have already been using that for a long time. (For instance, remember those media blurbs about lots of porn being found in the raid on bin Laden?) As such I'd bet they have infrastructure in place to detect such communications.

But in the same half-joking vein, some other problems:

1. I recall reading that such stegonagraphically watermarked videos were pretty easy to detect using simple statistical analysis of the noise in the frames. With dedicated hardware for multimedia decoding cheaply and ubiquitously available, frames can be efficiently extracted and the additional number of bytes does little to slow them down. If the hidden data is encrypted, well, you could just do that without the porn.

2. Unless you make it as easy as tapping a key to start communicating, it still does not remove a main barrier to widespread adoption of encryption, which is inconvenience.

3. The problem with porn is, people will just end up paying more attention to the medium rather than the message.

> you could just do that without the porn.

I think the idea is that it's something people naturally keep hidden, so it doesn't look suspicious to have handed to someone n a USB drive or sent privately. There's not really much suspicious about people hiding porn.

I read some research about provably secure steganography , which claim that if you know the statistics of the communication channel, you can build secure steno, at least to the level one way functions.

So it is interesting if in reality there is a stego detection tool.

www.cs.umn.edu/~hopper/tc-stego.pdf

Btw one of the authors is Luis von ahn, the founder of duelingo, the language teaching/translation app.

> if you know the statistics of the communication channel, you can build secure steno

That's a pretty big if though. I mean, if you miss one nuance the NSA didn't, then your scheme could be toast. At least in the high data limit.

You're right, that's a very big if.

Also, most of the research in the field has been done on black and white image, so it not directly usefull.

And come to think on it, if we're talking about image stego, what prevents the NSA from controlling the statistical distributions of output of cameras in such a way that stego will be noticed(even after some image transforms) ?

Maybe the answer is to have a program that adds low scale random noise to as many images as it can before upload to the internet.

It would be easy enough to make a one-click batch process, and would help provide cover noise for real communication channels.

Also, this group invented the CAPTCHA.

If I am not mistaken you mean "steganography" and not "stenography" as the latter is the person(s) that transcribe in a courtroom. I have never seen a spell check that had the word steganography in it though.

"I have never seen a spell check that had the word steganography in it though."

Are you sure you looked hard enough? Most of them include it as very faint morse code somewhere in the icon.

You're quite right. :)

Already being done. Watch more carefully ;) J/K

So it's now in the realm of maybe-to-probable that the NSA has every domestic phone call on tape as well as some non-trivial sized chunk of all emails sent? Plus for sure has all metadata they can get their hands on? Plus cable taps on most (all?) internet backbones?

Maybe SETI should try a SETI@NSA

Are you suggesting that NSA should wire-trap the aliens too? :)

Wireless-tap would be more appropriate.

Obviously we need to know where the eggs are getting laid.

I mean before we cure cancer, figure out how the brain works and solve inequality all the alien eggs must be found.

Plus anything municipal agencies are collecting. Eg. License plates, etc.

>We have turned intelligence into a regulated industry in a way that none of our allies, even in Europe, have done. We have all three branches of government involved in overseeing the activities of the NSA...

That a whistleblower had to reveal the programs, that Congress was largely unaware that any such programs existed, and that NSA personnel are content with lying to Congress about the programs pretty well contradicts that claim. Still, glass half full, I suppose it's somewhat encouraging that PBS is reporting on this.

Am I he only one who read that bit about wire-tapping the Supreme Court judges? While unsurprising, that really seems to cross the line from counter-whatever to political manipulation.

Absolutely. It's starting to play "God".

Holy crap:

RUSSELL TICE: ...The NSA were targeting individuals. In that case, they were judges like the Supreme Court. I held in my hand Judge Alito's targeting information for his phones and his staff and his family.

If true, that's... mind boggling. What possible valid reason could the NSA have to secretly monitor the calls of a US Supreme Court justice? And his staff, AND family?

I really hope this allegation is false.

I hope as well, but I doubt it. That kind of secret power begs to be abused. The system of checks and balances that we learned about in civics class is simply gone. A willful executive with these powers at hand can sway the outcome of any congressional vote, any Supreme Court decision, or any media story. Our immune system against tyranny has been compromised and irrevocably so. We haven't yet descended into despotism but it is inevitable once a strong enough individual rises. To carry the analogy further, it's 1984 and we've just contracted HIV.

They throw out the claim (~4:10) the facility in utah could store "about 100 years of the world's communications".

Has anyone done the math? How much could that facility store. If they were indeed recording word-for-word content, how many days/months/years could they store?

I think the metadata storage requirement is reasonable. These capabilities are already part of basic billing systems at the telcos, aren't they?

http://www.forbes.com/sites/kashmirhill/2013/07/24/blueprint...

Very interesting. The tl;dr seems to be: nobody knows for sure, but yes... it could totally be used for full content storage for some length of time.

No, it's not "nobody knows for sure." If you do the right math and don't assume that they have the storage technology not available to the rest of the world (which is a reasonable assumption) it's obvious that the data storage in an accessible way inside that center has an upper bound of an order of around one exabyte (1e18) (a million terabytes). So it's even more probable that it's less, say 0.2 EB.

Now somebody here claims it's around 1e7 B compressed for an hour of phone talk (not transcribed only compressed). That gives 2e10 hours stored. Storing it for 2e8 Americans there's space for 100 hours per person. But they also want the world, not only Americans.

You seem very confident of your input parameters. What are you basing 0.2-1EB on? It seems like there is a lot unknown about the utah data center, so all their "expert estimates" sounded more like guesses. Hence my conclusion "nobody knows for sure".

Even if, as you say, the estimate is 100hours per person. How much time does the average person spend on the phone? My current cell plan has 200 minutes per month, and I never go over. Say the average person spends 1000 minutes (16.7 hours), that means you could store a month of rolling phone conversations, and be only at 16.7% capacity. That leaves a fair bit of extra room for persons of interest internationally.

Your estimate is correct, somewhere on HN I've read that average young American talks 20 hours per month, older somewhat less. You're also correct that as soon as they decide that not everybody is actually important, the capacity to store communications of "persons of interest" is enough for "the whole life of the persons of interest."

Regarding my estimates, somebody here on HN also sent a link to the highest capacity automated tape libraries, and using them instead of hard disks the estimation of capacity still fits in the same order of magnitude. I still assume hard disks are more convenient than tapes for anything that needs to be accessed "when needed" (who would accept to say "will get the data bout the terrorists regarding the attack tomorrow, but it will take two days." So I can even bet that the main storage there are hard disks.

No, says an unnamed NSA "colleague" of a former analyst. Say what you like about Snowden, but at least some of his claims have been backed up by evidence. What have these guys got? Why not name names?

A stay in a penitentiary helped managed by SAIC[1].

[1] http://www.alanco.com/news_040104.asp

Because then they'll have to flee the country like that "traitor" Snowden?

Not saying it wouldn't be the right thing to do, but I am saying it would definitely come at a cost to them and their friends.

Why? A colleague of Tice's told him a few weeks ago that every domestic communication is being recorded. Tice is now a private citizen, and he has every right to tell us that colleague's name.

But he won't, and I suspect that the reason is that his claim is fanciful.

That's definitely possible. But it's also possible he doesn't want to out his friend because he's a decent guy. You know we would all think this guy was a scumbag if he outed his friend/colleague at the NSA who would likely suffer for it.

Without hard evidence it's hard to believe him, you're right. But not naming his source doesn't effect his credibility negatively in my eyes. It's equally likely he's not naming his source because he's a nice person, as it is because there is no source.

> But he won't, and I suspect that the reason is that his claim is fanciful.

Do you find this more probable than the fact that by giving out a name he'd set up his friend between a rock and a hard place?

If rights were really respected we wouldnt have this, now I am at loss for words so am settling with clusterF#$%, to begin with.

Recent history tells us there's proof out there, and that it typically follows a denial by Obama or the intelligence communities.

I get your emotional position, but this is just some bad logic. "If the government denies it, it must be true" is not a useful tool. How about "recent history says that this kind of thing could potentially be true, and that the government wouldn't admit if it was."

Thanks for the condescension, but you misunderstood me: we'll find out after the denial, when the doc release occurs. The denial doesn't substitute for the doc release.

"Why not name names?"

Because they can see what's happening to Snowden and Manning?

From my outside perspective, it seems Obama's "whistle blower" rhetoric was pure electioneering lies…

Persistent targeting, one way or another, by drones managed by SAIC[1].

[1] http://www.dailyfinance.com/2013/06/14/news-saic-wins-95-mil...

Digitally stalked due to dissent by for-profit "Domain Awareness Centers" run by SAIC[1].

[1] http://oaklandwiki.org/Domain_Awareness_Center

In many ways, hard evidence is the last thing a whistle blower wants. Its the evidence that scares the authorities. While its just "claims", you go free. You can be mocked and dismissed as a crank. If you have evidence, well, Snowden, Manning, ...

> He claims the NSA tapped the phone of high-level government officials and the news media 10 years ago.

That would explain why Obama seems to have become the best sheep ever:

1. Spy on them

2. Collect everything that's somehow "embarrassing" for them

3. Regularly remind them of what happens if they don't do what would want them to do

By that logic, they have probably "vetted" Obama thoroughly, before deciding to not discredit/destroy him completely in front of the public.

I truly believe this is what 'democracy' has become, not just, in the US, but the UK as well.

Indeed. This is the obvious issue every seems to be skirting over. Even the mass media seems not as out rages as it easily could be. Problem is we can never know to what extent this may or may not happen. What is needed is a high ranking politician to come out, admit to something awful and go on record to being black mailed.

See, I cant believe that you get to politician age and have a perfect record. I cant believe that the security services would be interested in politicians back grounds. And now we see the mechanism to get all the information they need to comprehensively check them out. So, I almost cant understand why a security service would not raise such issues with prospective politicians, high ranking business men, media, etc. It would be negligent not to. Well, with out saying opr threatening anything, such people know that the security services have something damaging on them which could be leaked. They don't need to be told, do they? Such people are fairly bright. The implications are obvious. Tow the line, don't dump on the security services. And if asked to support some evil new power, well........

To my mind the blackmail circle and potential to select politicians, and others is too obvious.

OF COURSE they'd store the contents of our communication, it would be foolish for them not to now that they have the means! The algorithms for speech to text are probably near perfect by now, especially since they have SO MUCH sample data to work with. I doubt you'd need to store the raw audio data, just turn the transcript into full text "metadata".

The main issue is that our information is being stored - INDEFINITELY. You've got to be naive to think this data won't be abused eventually. It will be sold, or it will be used to control. Reading between the lines in some of these news stories, I have to believe that all the walls between the civil and law enforcement agencies are coming down, and it's all one big data mart FTW.

We must bar them from generally capturing and storing information about us. A crime needs to be under investigation about an individual or group before the data capture can begin.

Backups! That's the upside of this whole NSA domestic spying brouhaha. How many times have you wanted to replay a phone conversation, or recover an email from a past account, or throw your boss's memo back in his face when he tries to "scope creep" your project?

That's right. Probably every one of us wishes for this capability. And then there's the federal budget deficit, no? I propose to kill two birds with one stone here. NSA can offer to "remember it for you wholesale" as Philip K. Dick put it! Maybe $0.10 per byte (or $0.25/minute for audio) to search and provide on CD virtually anything you have said or done since 1992.

It's actually rather comforting to think that, although they're not very good at their primary mission of stopping terrorism, they have this great ancillary benefit that we can all take advantage of!

Should this turn out to be true, the legal rationale is likely that a warrant is only need to read a communication, not to collect it. So "they" archive everything and then promise to look at it only when legal to do so. The NSA can even record American citizens if they promise not to read the data if it turns out to be an American they've recorded (unless the FBI wants the data, of course).

Even if this promise holds, though, people may dislike this policy, to put it mildly. If a robot takes a naked picture of me, it's an invasion of privacy even if no human ever looks at it.

If you are tired of having the same discussions over and over again:

https://news.ycombinator.com/item?id=6152935

he's making a list, checking it twice; gonna find out who's naughty and nice...

Pretty sure that's God.

Crazy idea: software that would generate TBs of fake communications (email, skype calls, etc) daily to disrupt NSA operations. Possible? Helpful?

This is not crazy. This has been done for centuries.

You send five messages that are not true and one which is, and have some way to know the truth.

If you're paying taxes in the US, you would probably hate this concept, b/c:

You spend something like 25% of your work life (taxes) to pay for something that works against you (mass surveillance). Then you start trowing stones at it while continuing to work for it.

Nah, doesn't make sense. According to this fighting nazism while being German wouldn't have make sense in 1930s because you pay taxes.

What I mean is:

We also need to start getting rid of the root of the problem, not only try to fight some symptoms (patching holes is OK for the short term).

Again, similar to Nazi Germany in 1930s. If you have a democratic society where 90% voters are against freedom and vote for all the wars are in favor of all the BS -- there is only this much you can do.

It turns out the Nigerian prince was a counter-NSA agent!

Reply on news.ycombinator.com