Documents shed light on U.S. surveillance programs | Hacker News Reader

Documents shed light on U.S. surveillance programs

cnn.com

53 pointsSuraj-Sun47 comments

Ah, yes. CNN. I noticed that when embassies in Yemen and elsewhere were recently shut down, CNN was careful to repeatedly note in its on-air and online coverage that it was thanks to the "intercepted" messages that we were aware of the threat. Ex: [1]

In fact, in at least one story, they went so far as to even specifically mention that the "controversial NSA program" was the source--apparently for those too slow to get the hint.

Such ridiculously transparent PR. And those brilliant and determined terrorists seem so easily thwarted, don't they? "Aww, rats. Foiled again! I knew I shouldn't have sent that email from my Hotmail account, with the subject 'Evil Terrorist Master Plan'. They closed the embassy. Now what do we do with these suicide belts?"

Really, who believes this stuff?

[1] http://www.cnn.com/2013/08/04/politics/us-embassies-close

I believe it actually. Evil and effective are not mutually exclusive.

The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying.

> Evil and effective are not mutually exclusive.

What is effective in this case?

I think effectiveness is open for debate as well. Consider the recent HN submission [1]. Without any information being released, we do not know if there was a credible threat, or if the NSA is simply delusional.

[1] https://news.ycombinator.com/item?id=6184984

>Evil and effective are not mutually exclusive.

So, we have instituted this massive security apparatus/surveillance state for groups of people who are largely incompetent, ineffective boobs?

But, we don't know that they are incompetent boobs?

Any way you slice it, it smells and begs the question: why are we doing this? And, in this particular case, it's a little too convenient that they are taking such public and hyper-publicized action on "threats" that do appear to be easily foiled. Because a.) the plotters allowed themselves to be intercepted even with so much recent publicity around our programs (btw, so much for the number one argument that whistleblowers like Snowden help the enemy) and b.) simply closing a few embassies nullified the threat. And, oh, BTW, let's make sure that our PR machine (read CNN and others) repeatedly mention that we all have the good ol' NSA to thank.

Sorry, I just don't see how this goes down easily for anyone.

>The real question though is if prevented attacks would have been prevented anyway without this evidence, or in general if the price is worth paying.

In this case, I don't believe there was an imminent threat. I think it was propaganda. But, on a broader level, I think your questions are valid for those inclined to believe we should trade freedom/privacy/etc for security (i.e. most Americans). But, surely the answer is in large part a function of whether and how many "effective" threats are really out there vs. just evil people.

So, your "real question" is unanswerable at this point on a general level. We don't know who the threats are and, in fact, they are intent on keeping this secret for "security reasons". It is the quintessential bogeyman. We must simply trust that these threats exist and are of a scale that warrant the massive security apparatus/surveillance state that we have erected.

It might not be pure opportunism, given that guerillas recently ate an entire embassy for lunch and the CIA has been taking a lot of heat for it. Even without the Snowden debacle they would be trigger happy about protecting assets using intel.

Quite possibly. Then again, you can probably find a modicum of truth in just about any situation.

"monitor" ... "internet traffic" ...

Great, you don't comb through every bit sent via YouTube, Netflix, etc. The smoke and mirrors are strong with this one.

It was YouTube's Robert Kyncl that was quoted in 2012 that "90% of all web traffic will be video." Add in your image traffic, and there's a lot of room to still "monitor" EVERYTHING of real value.

So true. I don't have the numbers but I guess 1.6% could be enough to monitor all email/messaging traffic.

Did some quick back of the napkin math.

Daily internet traffic (from the NSA white paper) is 1,826 Petabytes. 1.6% (* .016) is 29 Petabytes that the NSA "touches."

If the average email or chat is 75kb, then the NSA just admitted it touches 389 BILLION messages DAILY.

Also, the NSA says it only "reviews" .025% (* .00025) of the 1.6% it "touches." Small right? Well that means that the NSA "reviews...."

97 MILLION messages DAILY.

Assuming they probably don't check solicitations, I'm assuming they "touch" all emails, chats, etc. Right? And they "review" a non-trivial amount of them by quantity.

Math check? Please someone tell me I'm wrong.

Edited — I was off a decimal point, still a lot of messages daily.

96 Million. Not that that changes your point at all. You're not wrong.

97M https://www.wolframalpha.com/input/?i=.025%25+of+1.6%25+of+1...

Of course 1.6% could be about everything in terms of information that is out there, excluding raw non-meta data like bit torrent, mpeg, etc. One way of 'hiding' data for the NSA would be to embed it in these raw data streams with incorrect metadata.. would definitely harder to find that.

Edited.

Nm, I missed a demical point in a different spot. #fml, you are right. 97M.

Which doesn't account for ignoring email spam (72% of email traffic in 2012, according to Kaspersky [0]), John in accounting having 3 images included in every footer, attachments being copied in every reply, and the fact a big player like Google keeps its gmail-to-gmail traffic under a secure connection -- not plug-n-play monitering.

And if we hold the USG to their promise towards the Constitution, "world" traffic should largely comprise itself of "non-US" traffic. We can safely say then assume that non-US users are most definitely monitored. All of them. Everything.

[0] http://www.kaspersky.com/about/news/spam/2013/Spam_in_2012_C...

(Replying here because your other comment is too deep)

I believe you have it right. The NSA document gets it wrong by one digit (page 6):

http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_th...

(0.025/100) * (1.6/100) = 0.0004%, and not 0.00004% as claimed in the PDF.

How convenient that the actual NSA PDF makes an error in THEIR PR favor. Meaning that the NSA makes it look like they monitor even less than they do!

Who would have thought they would have made THAT convenient mistake.

Well, you don't know that. It could have been 0.16% instead of 1.6%. All we know is the math doesn't work.

Now, the only thing missing is the NSA's private dictionary that defines their interpretations of, "the world," "internet traffic," "monitor," "review," and so on. They have already established that they use these terms in highly specific and non-standard ways, so taking their word for anything automatically accepts their framing and leaves the word-taker out in the cold by definition. We aren't in the club, they don't want to tell us, so their numbers mean nothing.

And that's assuming that we believe them. When the EFF audits their operation and produces a number, I'll take it seriously. But if their director is willing to lie to Congress, I have no reason to think they won't lie to us.

This is a game of semantics. They only "monitor" communications that they are "lawfully authorized" to monitor. Every government official has been playing this game for 2 months now.

The real questions are:

1. HOW MUCH DOMESTIC INTERNET TRAFFIC IS COLLECTED?

2. HOW IS THAT INFORMATION ORGANIZED?

3. HOW LONG DOES THE GOVERNMENT KEEP THE COLLECTED INFORMATION?

Don't use "collected"; as they use it "collection" only applies when captured and stored stuff is looked at.

The NSA are semantic sociopaths. You can never expect them to answer a question honestly because they'll redefine each term to some unexpected yet valid meaning.

... or invalid. They're not that picky.

Indeed. And I would add one more: WHY?

Can't wait to learn how, like "collect" now means inspect, "monitor" now means permanently store.

Assuming storage, it's simple: metadata, deduplication, compression, selective monitoring, keeping only the logs but not the content (e.g. for YouTube videos), exclusion of certain bandwidth-intensive protocols, as 'thezilch mentioned...

Aaaah, the confusing and deceitful language that is doublespeak.

Where is the white paper?

edit: http://www.scribd.com/doc/159222203/NSA-White-Paper-on-PRISM

edit_2: http://www.scribd.com/doc/159218355/Obama-Administration-Whi...

http://i2.cdn.turner.com/cnn/2013/images/08/09/2013_08_09_th...

CNN's link for those who dislike Scribd because it hates us all. :)

> "In any case, the Justice memo says "any arguable privacy intrusion arising from the collection of telephony metadata would be outweighed by the public interest in identifying suspected terrorist operatives and thwarting terrorist plots."

A little footnote at the bottom is all this is worth? How do they know this? How do they know that "any arguable privacy intrusion" is outweighed by the public interest in identifying "terrorist" plots, when no one was given the choice and no debate has occurred? That was (allegedly) Snowdens' entire point of releasing the information that he did!

Not to mention that the premise isn't as simple as "privacy intrusion vs thousands of people getting blown up".

because they debated this very thing before it was passed into law. it was public as well. perhpas you heard of the PATRIOT act?

For the record, if I ever get blown up, I don't want to be used as an excuse for stripping citizens of their liberties. My death is inevitable; freedom isn't. We thought it worth paying in blood to get that freedom, and I think it even more valuable today.

And honestly, as a citizen, I see myself as partly responsible for a lot of the things that drive terrorist actions against the US. I know that if somebody blew up my family's wedding, I'd want revenge. That's not to say that I'd be happy to be killed in retaliation for some US government idiocy. But if the cycle of revenge violence has to stop somewhere, let it be with me.

So, what's the interesting traffic left which can be passively monitored?

We absolutely need OTR and then something like OTR for multiparty chat for chat. And, ideally, traffic analysis resistant. It needs to be the default, and painless.

Email is mostly a lost cause for TA, but TLS covers a lot of transport. Sometime in 2014, TLS should be the default, and "required TLS" should be considered an acceptable way to configure your MTA.

Standard web traffic is increasingly moving to HTTPS for a variety of reasons unrelated to NSA.

The really hard part is protection from metadata collection, and making big targets resistant to demands for 1) disclosure and 2) modification to accommodate monitoring.

Considering how much of the internet traffic is youtube and netflix (almost 50% according to http://www.reelseo.com/netflix-youtube-us-internet-traffic/) and how much is just duplicated (specially in the big things like very shared p2p files) and things they already had, that number don't take out the possibility that they are taking all the interesting/private stuff that goes around.

Also, that don't includes what is not internet traffic, like intrusion into private networks and servers.

So that place in Utah is gathering only 1.6% of ... whatever it is gathering.

CNN and a government is a kool-aid for the masses.

What percentage of the internet isn't passive content, like Netflix, other streaming media? Then subtract out garden variety Web content. Then subtract spam.

I suspect 1.6% is pretty close to the amount of interpersonal e-mail and real-time voice and text that traverses the Internet.

Reddit x-post from /u/Jou_ma_se_Poes in this thread:

https://pay.reddit.com/r/worldnews/comments/1k0qwn/two_encry...

Everyone should read this, if you're interested in this story at all. Great succint capture of the currently known info:

---

Knowledge is power. The NSA haven't been to careful with their information. With 15000 documents classified documents out there I am sure that eventually my "unknowns" for the programs below will be known. I hope someone at the NSA chokes on their coffee when they read what we know and what we will still know. I am sorry the list is so long, the NSA have been busy. If you know what any of the unknowns are do let us all know.

Dial Number Recognition (DNR) Digital Network Intelligence (DNI) Geospatial Metadata Analysis (GMA) SIGINT Geospatial Analysis (SGA) SIGINT Terminal Guidance (STG) Each SIGAD is basically a collection site, physical or virtual; the SIGAD alphanumerics are used to indicate the source of intelligence FOR a particular report.

The NSA listening post at Osan in Korea has the SIGAD USA-31. Clark Air Force Base is USA-57. US-987LA is with Bad Aibling Germany US-987LB are telecommunications surveillance in Afghanistan

XKEYSCORE processes most of the SIGINT traffic that comes into the NSA by way of various SIGADs, and compartmentalizes it by selector. A selector might be "RUSFOR," which would stand for Russian foreign ministry intercepts.

PRISM is US-984XN analyze information collected and deposited in a database - 117,675 active surveillance targets in PRISM's counterterrorism database

Analysts working on a problem can request that a particular collection site be tasked, or used. The form they fill out is known as an SP0200. Additionally, when they wish to discontinue using a SIGAD for a mission, they send in another SP0200.

TURBULANCE - Consists of nine sub programs including Traffic Thief, Turmoil and Tutelage. Other subprograms unknown.

TRAFFIC THIEF - a database of "Meta-data from a subset of tasked strong-selectors" An example of a strong selector is an email address. In other words, it would be a database of the metadata associated with names, phone numbers, email addresses, that the intelligence services are specifically targeting

AGILEVIEW: DNI tool

AGILITY: DNI tool

SKYWRITER: DNI reporting tool

TREASUREMAP: DNI visualization tool

DISHFIRE: DNI

ETHEREAL: DNI

TUNINGFORK/SEEKER: DNI

OCTAVE: DNI/DNR tool for tasking

UTT - DNR tool for tasking

MAUI: a database for finished NSA intelligence products.

PROTON: SIGINT database

CULTWEAVE: SIGINT database

HERCULES: CIA terrorism database

MAIN CORE: database which contains personal and financial data of millions of U.S. citizens believed to be threats to national security

PINWALE/UIS: archived foreign and domestic e-mails database (FBI access)

MARINA - Internet/Telephone metadata database Unclear

MAINWAY: Call records Database - contains metadata for hundreds of billions of telephone calls made through the four largest telephone carriers in the United States

FALLOUT: Filter for MARINA and MAINWAY

NUCLEON: Voice data Database (FBI access)

CONVEYANCE: Filter for NUCLEON

ARTEMIS: Geospatial analysis

AIRHANDLER Geolocation analysis

DRAGONFLY: Geolocation analysis

GJALLER: Geospatial analysis

GOSSAMER: Geospatial analysis

GROWLER: Geospatial analysis

KINGFISH: Geospatial analysis

STINGRAY: Geospatial analysis

TEMPTRESS: Geolocation analysis

WITCHHUNT: Geolocation analysis

CADENCE/GAMUT: Collection mission system for tasking

UTT (Unified Targetting Tool?)

COURIERSKILL: Collection mission system

TURMOIL: Collection mission system

METTLESOME: Collection mission system

WEALTHYCLUSTER: Collection mission system

Enhanced WEALTHYCLUSTER (EWC)

MASTERLINK: Tasking source

HIGHTIDE/SKYWRITER: Desktop dashboard

RENOIR: Visualization tool

SKOPE: SIGINT analytical toolkit

DROPMIRE - NSA surveillance programme aimed at surveillance of foreign embassies and diplomatic staff, including those of NATO allies.

BLARNEY - program to access internet data at key junctions and is facilitated by arrangements with commercial cable companies and internet servce providers. (part of Upstream)

FAIRVIEW - NSA mass surveillance programme aimed at collecting phone, internet and e-mail data in bulk from the computers and mobile telephones of foreign countries' citizens. (part of Upstream)

BOUNDLESS INFORMANT - is a big data analysis and data visualization system to give NSA managers summaries of NSA's world wide data collection activities - summarizes data records from 504 separate DNR and DNI collection sources (SIGADs).

CPE (Content Preparation Environment) is a tool for writing report based on the analysis.

ANCHORY: an intelligence community-wide database for intelligence reports.

Mira4 and VERAS: German programs obtained by NSA (purpose unknown)

Surveillance Detection Units (SDU) are organizations belonging to the US Government that have conducted secret surveillance that potentially broke national laws in various European countries.

SIMAS - SDU's data is collected in a system called Security Incident Management Analysis System

NSANET - network carrying information rated Top Secret or Sensitive Compartmented Information. (Snowden access)

TERRORIST SURVEILLANCE PROGRAM - forerunner of PRISM

STELLARWIND is the code name of a Sensitive Compartmented Information security compartment for information collected under the President's Surveillance Program

TRAILBLAZER - earlier NSA effort.

PIONEER GROUNDBREAKER - IBM, ATT and NSA...

TEMPORA - British program including recordings of telephone calls, the content of email messages, Facebook entries and the personal internet history of users. Shares this with the NSA.

CCDP - The Communications Capabilities Development Programme is a UK government initiative to create a ubiquitous mass surveillance scheme for the United Kingdom. It would involve the logging of every telephone call, email and text message between every inhabitant of the UK

DHS

FAST (Hostile Intent) - Future Attribute Screening Technology - precrime screening

MATRIX - Multistate Anti-Terrorism Information Exchange Program closed down 2005

Trusted Traveller Program https://www.dhs.gov/trusted-traveler-programs

No Fly List - 22000

Terror Watch List - over a million

Disposition Matrix - number unknown

Programs unknown in the NSA:

AIRGAP/COZEN - unknown

AMHS - unknown

ARCANAPUP - unknown

ARTEMIS - unknown

ASSOCIATION - unknown

AUTOSOURCE - unknown

BANYAN - unknown

BEAMER - unknown

BELLVIEW - unknown

BLACKPEARL - unknown

CADENCE - unknown

CHALKFUN - unknown

CINEPLEX - unknown

CLOUD - unknown

COASTLINE - unknown

COMMONVIEW - unknown

CONTRAOCTAVE - unknown

CONVERGENCE - unknown

CREEK - unknown

CREST - unknown

CROSSBONES - unknown

CYBERTRANS - unknown

DOUBLEARROW - unknown

FASCIA - unknown

FASTSCOPE - unknown

FOREMAN - unknown

GISTQUEUE - unknown

GLAVE - unknown

GLOBALREACH - unknown

GOLDMINER - unknown

GOLDPOINT - unknown

GOSSAMER - unknown

HIGHTIDE - unknown

HOMEBASE - unknown

INFOSHARE - unknown

INTELINK - unknown

JOLLYROGER - unknown

LIQUIDFIRE - unknown

MASTERSHAKE - unknown

MESSIAH - unknown

METRICS - unknown

MICHIGAN - unknown

NEWHORIZONS - unknown

NIGHTSURF - unknown

NORMALRUN/CHEWSTICK/FALLENORACLE - unknown

OAKSTAR - unknown (part of Upstream)

OCELOT - unknown

OCTSKYWARD - unknown

PATHMASTER/MAILORDER - unknown

PANOPTICON - unknown

PLUS - unknown

PRESENTER - unknown

PRINTAURA - unknown

RAVENWING - unknown

ROADBED - unknown

SCORPIOFORE/CPE - unknown

SIGNAV - unknown

SHARKFINN - unknown

SNAPE - unknown

SPOTBEAM - unknown

STORMBREW - unknown (part of Upstream)

STINGRAY - unknown

SURREY - unknown

TAPERLAY - unknown

TAROTCARD - unknown

TRACFIN - unknown

TRAILMAPPER - unknown

TREASUREMAP - unknown

TRICKLER - unknown

TUNINGFORK - unknown

TUSKATTIRE - unknown

TWISTEDPATH - unknown

WEBCANDID - unknown

YELLOWSTONE/SPLITGLASS - unknown

Here is the second part of the post as it was too long for HN:

-----

Edit. WIRESHARK and ARCMAP removed. See Entropius' post. Both commercial/harmless programs. Don't want to create a paranoia sandwich. It is bad enough as it is.

Second edit ETHEREAL removed - thanks to PointyOintment and azaydius

Third edit: Someone wrote to me asking what my problem is:

Thank you for contacting me with your concerns. I will answer as honestly and faithfully as I can. I am not an American. I am an attorney in a far away land. Look up my user name if you really must.

ALL the information I have posted is publically available. ALL of it. The NSA should be more careful.

What has the information been used for? The one XKeyscore slide stated over 300 terrorist captured. The NSA claim that their internal spying has disrupted 58 plots in the USA. Where are all these supposed terrorists and when are their trials starting? We're not talking about the Boston Bombing, because they weren't there for that. We're talking about 58 seperate terrorist attacks which Americans have never heard about? Oh wait I hear you say "they were kept secret for a reason..." No... How about this is all fiction. Where is the evidence? General Alexander should be able to sit back, smile and start listing the targets, preparations and everything regarding each of these "58 supposed plots", yet he can only point to a single guy sending $8500 to Al-Shabaab. WOW! $8500!!! Stop the frikken presses! $8500!! /s

The other uses of internationally intercepted communications is to mark people out for death based on who they have been talking to. You are aware that the US uses telephone metadata from Afghanistan to launch strikes. Meaning that the content of the phonecalls matters not, what matters is how many times you speak to someone considered to be few enough hops away from someone considered a terrorist? I think the continual drone strikes are a crime against humanity and you only have to consider what it is like [to live with the constant threat of death hanging over them.](http://www.livingunderdrones.org/living-under-drones/)

I'm not really enjoying my schadenfreude and I told you so when it comes to US citizens losing their rights. FFS! I TOLD YOU SO! You know how many thousands of downvotes I have had to endure trying to warn of the dangers? I eventually just gave up. Americans just knew better. It fucking pains me that I got nowhere fast.

I believe that privacy is a basic human right. It is not permissable to trample it underfoot because "terrorists" and "terrorism". I have lived through a real armed campaign with bombings and landmines and I didn't buy it then and still not buying it now.

Which USG clandestine operations do you believe are of value? It all seems to have backfired rather spectacularly.

What about the abuses of these very same progams that we're hearing about? Do you think the internal surveillance should be automated and watch for people spending money they shouldn't have, can't explain and/or haven't declared to the IRS? So you think the internal surveillance should be used for petty drug possession busts?

How many people are there in Al-Qaeda? A 100 as the ISI reckon? A 1000? How many of those are in the USA? OK. Now WHY are there 117,675 active surveillance targets in PRISM's counterterrorism database? It's out by a factor of 117:1. There are over a million Americans on the Terror Watch List. What is THIS ALL ABOUT? Do you have any idea?

What is the FISA court record for and against the government? All those warrants to do all that spying.... how many cases have eventuated because of this.

So yes, I am against the NSA and everything they do and enable. I don't think the NSA represents the USA in any meaningful way. From the outside it looks like a rogue operation with no meaningful oversight.

In other news, 98.4% of the world's Internet traffic is Nigerian spam.

If they had any clue they'd just watch traffic to Google Analytics and instantly have something like 75% coverage. Or maybe they already do that.

And what percent of those 1.6% may involve an American citizen? And of that percent, how many are being collected under a lawfully issued warrant?

Yeah, it matters what they do to Americans, but what matters more is what is done to the rest of the world's stuff and people. We don't get to vote in the thing which gets a president in (and what an interesting process that is). We get no say, but its apparently ok to bomb us, drone us, spy on us, grope at us, render us, irradiate us to see us naked etc. I supposed it helps keep me from terrorising Americans though.

Also of note: NSA claims that analysts review 0.00004% of global Internet traffic (0.0025% of the data they monitor, by my math).

so the NSA analysts actually review "only" 7.304 TB (1826 PB * 0.016 * 0.00025) of data daily...

... and to "monitor" the 1.6% traffic, they must filter the data at a (combined) rate of 2.705 Tb/s to keep up.

I would never trust NSA again EVER.

Wow, a 98.4% failure rate.

/sarcasm

That's still huge amount, no matter what it actually means.

If they ignore most of bittorrent and streaming services that dominate Internet traffic, the percentage of mail, skype and http traffic they monitor is incredibly high.

Reply on news.ycombinator.com