XKeyscore: NSA program collects 'nearly everything a user does on the internet' | Hacker News Reader

XKeyscore: NSA program collects 'nearly everything a user does on the internet'

theguardian.com

1,688 pointssinak612 comments

This is overwhelming. Even when you always hear the claims about we knew this was going on, somehow it is still shocking when you see it all laid out infront of you with screenshots and the capabilities described.

I can see how they get HTTP information, since they would intercept at transit hubs - but how are they getting all Facebook private messages and Gmail?

I was also looking for another unique ID that users are identified by - perhaps a machine or browser fingerprint or some form of intel that can 'glue' different browsers together and make a best guess if they are the same person (Facebook does this with device and user cookies) but couldn't find anything. It seems they rely solely on email addresses, IP addresses, cookies and HTTP headers.

So if you are browsing via 16 tor circuits and a browser that defaults to incognito with session histories being wiped, they couldn't reconstruct your history.

Users of PGP/encryption products being singled out is terrifying. The sooner we have the whole world using decent encryption tools, the better.

Edit: Gmail messages must only be captured when they leave the Google network. They are the only provider to support server-to-server TLS: https://twitter.com/ashk4n/status/346807239002169344/photo/1

They must only be getting a slice of the Facebook chat data, since the transport there is also https.

Facebook Messenger, on the other hand, uses MQTT, so it transmits and stores in plaintext. It has support for encrypted + signed messages with OTR if you are using an alternate client such as Adium or Pidgin.

Really need to go out an audit all of these services and let users know which are better.

Assume that Tor is broken. With this level of deep network monitoring, low-latency onion routing is essentially useless.

Hidden services are still secure, presumably, because there is no exposed section of the network to inspect. All they can do is monitor and do statistical analysis, and maybe mess with the traffic to try to get more ideas of flow.

I wouldn't for a second bet on it. A hidden service has exactly the same issue as traffic that exits the network. The topography looks like this.

    httpd > tor node > tor node > tor node > rendezvous point < tor node < tor node < tor node < client

With enough monitoring, the location of the web server (or other hidden service) can just be found out by bombing the hidden service with traffic and seeing what end point lights up with traffic. With fine enough monitoring you wouldn't really need long to find out the real location of the server. It's just not something the network can effectively hide, even if it used chaff (padding) to hide the wheat.

There's practical attacks for enumerating hidden service public keys, and so I wager that there's somebody somewhere with a complete map of the real server locations as well.

According to tor metrics only 17% of tor endpoints [1] and a similar percentage of relays [2] are in the USA. The kind of monitoring you propose would require a much higher portion of them to be under NSA control.

[1] https://metrics.torproject.org/users.html [2] https://metrics.torproject.org/network.html?graph=relaycount...

Who said the NSA had to limit its nodes to the United States?

The NSA seems to have extensive monitoring in many countries, not just the United States.

The question isn't how many endpoints the NSA has, it is how much bandwidth they have at the endpoints (actually, it is more about how many unique users use their endpoints). But, assume that 1% of Tor connections goes through an NSA exit node. 1% of that 1% would go through both an NSA exit node at both ends, and is therefore comprimised. Tor tries to mitigate this by always using the same exit nodes for your connection (reducing the chance of ever being compromised, but if you are compromised, it is for much longer). However, inevitably you occasionally do need to change your exit nodes, which gives the NSA another roll of the dice. Additionally, when talking about drag-net surveillance, 1% of 1% is still a lot.

The bigger protection is the ease with which the NSA can mount this attack on TOR. I have no doubt that they could do it, however I do question if they can do it on a massive scale.

I imagine that when you have taps at all the colocation centers (which each node would need to go through - and even a surprising number of hops overseas go through the US due to the cheaper price of bandwidth) you may not need to control the endpoints to break anonymity, with enough statistical analysis of the packets entering and exiting the known tor nodes. Tor doesn't work against attackers who can monitor the whole network, and the developers say so up front.

"Tor tries to mitigate this by always using the same exit nodes for your connection"

Think you're getting your entry and exit nodes mixed up there. Tor chooses a small number of entry nodes (entry guards) and attempts to only use those.

I would think Silk Road provides enough incentive that if the government could defeat tor hidden services, they would have busted Silk Road.

If every police officer had access to these tools, the news would leak much sooner.

So I would think these tools are available only to a select few, and those are more interested in more high-profile tasks like catching extremists or going after political opponents.

I, frankly, don't think SR is that high on government list. Not yet.

Absolutely not. The government is not one unitary piece. The NSA is not the ATF is not the FBI. These capabilities were likely kept secret from other governmental agencies as much as the public.

Furthermore intelligence agencies are well aware that every action communicates information back to their adversaries. It's a no-brainer to let Silk Road exist if you think doing so gives you the edge on terrorism, or otherwise furthers the national interest.

Silk Road is a few pennies and few gram transactions. [See the data here http://arxiv.org/abs/1207.7139] It would be foolish to expose their snooping capabilities for this, right? Wow, Tor is not considered safe... Amazing

No way. What you forget is that once they bust it -- then they've REVEALED that they have the capability to do that.

Once they've revealed that, then people take account of it, and it becomes harder for the NSA to monitor them.

Half of the signals intelligence game is keeping your capabilities secret, so you can keep monitoring the signals, rather than have your target change their game.

That is to say, if they can get into Silk Road, then they probably ARE already monitoring everything that happens on Silk Road, and they'd rather it stay UP so they can keep monitoring the people on it (being very careful never to reveal that they can monitor it), then bust it so the people go elsewhere.

Historically, it has proven unwise to underestimate the NSA. Not being a booster, just thinking about Keyhole, tapping submarine cables, etc., etc.

Thanks for pointing this out. Good to see a peer preacher.

How long has https been an option with Facebook and messages? I don't think it was always required, if ever.

It's only been required in the last year or so, prior to that it was an optional extra in the security section.

Connections secured with TLS aren't effective if a) you can compromise the CA, b) have the private keys, c) have cooperation of the appropriate company (most likely), d) have compromised the server, e) are aware of flaws in the encryption algorithm, f) weak keys have been used, or g) have compromised the client computer.

It's also not effective if h) TLS was never used in the first place. Facebook hasn't always been all that secure to eavesdropping.

Compromising the CA isn't as powerful as most would think. It does allow you to MITM, however it does not allow you to do so invisibly. Someone who is paying attention to the public key could notice that it changed.

But you could do it for a specific target and that target has a high chance of not noticing. Doing it indiscriminately on the other hand...

> how are they getting all Facebook private messages and Gmail?

Perhaps you missed the news about PRISM? :)

This presentation is from 2008. According to the presentation on PRISM Facebook joined the program on 3 June 2009. That would indicate that the searches here are based, most likely, not on participation by Facebook but by passive sniffing of HTTP traffic and then session reconstruction.

In 2008 Facebook ran on HTTP, so back then it would have been easy to sniff this data. I believe Gmail also transferred in plain text back then. When those companies switched to HTTPS, the NSA likely 'leverage some pressure' to get them to join PRISM, which puts the data back in this system.

You've got your timeline wrong. Facebook didn't support HTTPS for anything but user login until 2011.

[0] https://www.facebook.com/blog/blog.php?post=486790652130

> That would indicate that the searches here are based, most likely, not on participation by Facebook

Gag warrants existed before PRISM.

EDIT: "National Security Letters"

Around about the time when people started rolling out SSL as standard. That'd make sense, as they'd need to move their beam-splitters (prisms!) to behind the SSL endpoints.

I think PRISM is just the public-private partnership aspect of this, where they have to go to service providers and install kit, as they can't tap SSL traffic.

From the screenshots it's obvious that the captured data is an HTTP form submission in facebook.

So they didn't have access to private messages, they just intercepted internet traffic and relied on it being unencrypted. Facebook didn't always enforce https by default like it does now

PRISM allows them to retrieve individual users' messages via a FISA court order. It doesn't allow analysts to instantly obtain private data for any user they want. :)

With regards to the data collection, the thing to realize (which I did so myself) is that email truly is the glue that ties together most internet services.

Take facebook for example. By default, almost any and all activity on the site is catalogued for you by email -- for your convenience. Someone mentioned you in an update, you get a notification. A friend sent you a private FB message, you can an email notification with the content in line (even with the support of replying to message via email as well).

Now, because email traffic on the internet is not encrypted by default, one is able to piece together the contents of communications just by looking at the email.

Essentially anything that you receive via email (e.g. password reset links; credit card statement summaries etc) is subject to capture and analysis. Given this, it may make sense to perhaps disable (potentially sensitive) email notifications as a workaround around this particular collection method.

The main thing that this new release reveals is not the scope of the data collection, but confirmation that analysts are given free reign to perform queries. Until this, there was an outside chance that the system required all database queries to be signed by a Judge prior to execution. This is not the case though; all queries are processed immediately, with essentially nothing more than a repo commit message as justification, and basically any analyst can do it.

Exactly. There were a lot of people from the government that came out in the past few months and said there are checks and balances and a lot of oversight in these processes. That clearly isn't true.

It will be interesting to go back through all of those statements with this new information/evidence on hand.

Greenwald has timed this well. He put out enough information early on to give Snowden opponents enough rope with which to hang themselves.

And if his comment further down in the thread is anything to go by then there is a lot more to come.

It's an interesting problem for the talking heads: How much will be revealed? They're caught between a rock and a hard place, if they start telling the truth they might reveal something that the leaked docs don't support, but if they tell a lie they might be found out.

This trickle strategy is working very well. The best cause of action for the people under the microscope would be to shut up and if they are compelled to talk to say the absolute minimum but to still tell the truth.

It's pretty impressive how Greenwald, Snowden et al are organizing the staggering/trickling. They're not just releasing any old info at periodic intervals. They seem to be anticipating the responses NSA/USG will give to particular leaks (e.g. analysts can't run searches, there are checks and balances) and choosing next leaks based on how they can prove those NSA/USG statements wrong.

It's like the Socratic method for public/government relations.

The goal seems not just to be exposing the magnitide of this surveillance system, but also the government's systemic disregard for public mandate in the USA right now.

>Greenwald has timed this well. He put out enough information early on to give Snowden opponents enough rope with which to hang themselves.

I have to wonder if the staggered deployment of the leak has anything to do with savvy, or more with his own need to digest what he's got as he works through it and reports as he goes.

Either way, the story has more legs than past revelations, so I'm happy for that, and I certainly would love for it to be the case that there is a degree of effective calculation behind the deployment of the info with the goal of keeping the conversation alive and neutering critics. Goodness knows that this story needs all the help it can get. It's up against not only the resources of some of the most powerful governments on the planet, but also the lacking attention spans of their populations combined with relatively disinterested media.

I'm heartened that the noise level has remained so high since the first Guardian article (in this latest series).

Response from Greenwald in the comments:

Q: Thanks for reporting this. I have to ask though, why is it that you are doling out this information now after the recent congressional inquiry into NSA spying and not earlier?

A: We've published almost two dozen exclusive articles about NSA spying in the last 7 weeks, in multiple different countries around the world. Is that pace not fast enough?

There are thousands upon thousands of documents and they take time to read, process, vet, and report. These are very complex matters. On top of everything else that has to be done with these articles, from explaining, debating and defending them in the media to dealing with the aftermath.

People can accuse us of many things. Not publishing enough or fast enough is hardly one of them.

That House vote was about one specific topic - bulk collection of phone records - that this newest article has nothing to do with. That House vote isn't the be all and end all: it's just one small battle in what I can assure you will be a sustained and ongoing discussion/controversy.

There is a lot more to report still. Accuracy is the number one priority. That takes time.

Thanks for the context Happer.

Devils advocate here: If in fact all of this is being collected, is it actually illegal to search without a warrant? If all of the above items are being siphoned off the internet via taps in concentrated NAPs around the USA and the world, and everything is in plaintext, this doesn't seem to be technically against the law.

Read the 4th amendment.

I think it would be very easy for the NSA to obtain the private keys to SSL of popular sites, eg Facebook, Gmail. Either through coercion or hacking.

Facebook ex-CTO joined NSA not too long ago, that might have helped?

Probably not as much as FB co-founder offering his company Palantir services to government on people's intelligence.

I also suspect that Palantir is involved.

Let's not forget about https://en.wikipedia.org/wiki/Narus_(company)

And some proof: http://www.haaretz.com/news/diplomacy-defense/1.528529

Haha, suspect. You know their tool for importing new types of data into a Palantir system is called Prism, right? Aggregating data from different sources and linking it is all they do.

> edit: Gmail messages must only be captured when they leave the Google network.

It seems easier for the NSA to tap datacenter <-> datacenter fiber links inside Google's network.

Why worry about decryption when you can have Google's frontend servers do it for you?

Why would Google (or anyone) link to them directly? with fiber no less! this stuff is alarming enough no need for FUD.

This XKS business seems about intercepting non-encrypted traffic as the references to HTTP payload quoted in the article would suggest.

(Keep in mind these slides are 5 years old, before Google Search over SSL)

The NSA has clearly tapped trans-oceanic fiber -- why not also tap high-volume inter-datacenter links?

I guess they could tap the fibre at a relay point, maybe somewhere like Hawaii.

> Why would Google (or anyone) link to them directly? with fiber no less! this stuff is alarming enough no need for FUD.

Who says Google has a choice or is even complicit? The backbone providers have mostly stayed mum and it's known that the likes of AT&T split their fiber for the NSA. If we're willing to go to the bottom of the ocean to tap fiber lines it's pretty easy to believe that we'd tap terrestrial lines too.

My understanding is that internet firms enjoy slightly more leverage, and that is why in contrast to telecos they are now petitioning the courts to reveal the scope of the orders.

That's all hand-waving. The courts won't allow it, the giants know that, so the internet giants use it as a chance to look good. Furthermore, it benefits the NSA for us all to think that Google, Yahoo, et. al., are not in their pocket.

> Why would Google (or anyone) link to them directly? with fiber no less!

"They have no direct access to our servers"

I wonder what a beam splitter consists of. Oh. A PRISM.

Beam splitters are, in general, not prisms. A prism, as traditionally referred-to (and in the NSA PRISM graphic) separates light of different wavelengths. In a signal tap, you want to split the intensity, not the wavelength. In simplest form, telecom signals are at a single wavelength; passing it through a 'Dark side of the moon' prism will only deflect the beam, not split it.

When one refers to a beamsplitter, it's usually a partially silvered mirror.

http://www.thorlabs.us/newgrouppage9.cfm?objectgroup_id=914

If it's fancy, it might use an evanescent wave to do the coupling, as in some cube beamsplitters.

Beamsplitters for optical fiber are more generally referred to as 'couplers' and involve bringing two fiber cores close enough for a long enough distance that the probability of coupling light from one to the other is the desired amount.

http://www.thorlabs.us/newgrouppage9.cfm?objectgroup_id=374

It is possible to split beams with a birefringent prism, but it is much less common.

http://www.thorlabs.us/newgrouppage9.cfm?objectgroup_id=745 http://www.thorlabs.us/newgrouppage9.cfm?objectgroup_id=917

Disclaimer for the following: I only work with optical fiber couplers occasionally, and not for telecom. Someone who works on telecom fibers daily will be more informed.

In summary, if someone wanted me to tap an optical fiber, I'd call up ThorLabs, get a matching coupler shipped overnight, cut the relevant fiber, slap APC ends on the fiber ends, and jack in. Splitting the beam in free space (outside of a fiber) with a prism is far more errorprone, unstable, and no more efficient. A fiber coupler has no moving parts, can't break, and won't take down a telecom's trunk line if someone breathes on it funny.

If they're actually using a prism, it's because of some sort of impedance/reflection minimization scheme; I can't conjure one that would work better than using simpler techniques though.

You can/do/might use actual prisms for a variety of reasons, however, such as if you're trying to get a frequency-multiplexed set of signals off a single fibre broken down as their constituent components - i.e. bulk data collection from a single tap on a mass fibre bridge.

Anyway, you're probably right, it's probably just bog standard parts, and PRISM was a buzzword for management.

Are couplers detectable via time domain reflectometry in practice?

Actually it's: "There is no free-for-all, no direct access, no indirect access, no back door, no drop box."

http://www.guardian.co.uk/technology/blog/2013/jun/19/google...

And I tend to believe him.

Or, the NSA could force Google to provide them with private keys to their certs for "national security" and then they can mitm all they want.

Why assume that inter-datacenter links are not encypted?

> Gmail messages must only be captured when they leave the Google network. They are the only provider to support server-to-server TLS

We should start lobbying for broader support for server-to-server TLS with perfect forward secrecy. While it alone is not sufficient to prevent the wiretapping of targeted individuals, it still makes fishing expeditions or "Big Data" level surveillance much harder. It would help keeping ordinary users' emails protected on the wire and secure the meta data of PGP emails.

> I was also looking for another unique ID that users are identified by

They can use plugins / extensions installed. Fonts installed. If cookies are enabled or not, etc. Check out: https://panopticlick.eff.org/

Most of the identifying information used by panopticlick requires using javascript/flash/java to obtain. As such, it isn't available when simply parsing HTTP headers and packets (as much of the data in XKeyScore appears to come from).

(That is, unless you visit panopticlick.eff.org, which then sends all of the processed information over the wire in the clear...)

Just want to clarify something:

> I can see how they get HTTP information, since they would intercept at transit hubs - but how are they getting all Facebook private messages and Gmail?

I don't know how they're getting GMail(and this is probably a slide from when GMail was accessible via HTTP and not HTTPS), but Facebook chat specifically is done over a non-secure XMPP server. The only 'secure' part of that transaction is login, as far as I remember, once you're past that none of it is encrypted.

With Gmail, all it takes is one request to almost any Google service to leak through a non http connection and they have your Auth cookie. Once they have that, they are you. And yes it is that easy, anyone can pull it off at Starbucks, hotels, even some ISPs.

Sorry for being so naive... does that cookie expire eventually? I have been using HTTPS everywhere on my machine, but if I log in to my Google account for YouTube, for example, from someone else's computer, how much data can they realistically download and how long would they have that ability?

Not speaking for Google, but in general, auth cookies (rather than identity cookies) will only be sent over HTTPS using the "Secure" cookie attribute. This is something done at the browser level, so short of using a very badly behaved browser or HTTP client, this is unlikely to happen.

You're right the slides are pre default HTTPS gmail (2007/8).

But even then gmail is the only webmail service that offers server-to-server encryption, so data can still get intercepted when communicating with someone using yahoo mail or hotmail for example: http://news.cnet.com/8301-13578_3-57590389-38/how-web-mail-p...

Good point.

You think HTTPS keeps you safe? All it takes is ONE recipient to have an insecure connection and the entire thread is revealed.

Isn't it nice how every email conveniently includes a copy of the entire preceding conversation.

>This is overwhelming. Even when you always hear the claims about we knew this was going on, somehow it is still shocking when you see it all laid out infront of you with screenshots and the capabilities described.

It has become a bit of a pet peeve of mine recently to see self-aggrandizing comments from users around the net about how "we should have known" and "none of this is new."

I'm a practically addicted news junkie (especially tech news) and while I've been aware of a fair amount of what has been exposed in this latest leak, it seems that every day there are revelations new to me, and what is revealed absolutely shocks the conscience. And I'm an outlier. I'm more plugged in to reporting on this subject than 99% of the globe's population, and this subject tangles with the rights and treatment of a large portion of the population of said globe.

The staggering majority had no clue, has no clue, and no, they were never informed. For all intents and purposes, the global media has been asleep or complicit.

It's staggeringly important to keep telling this story at every level specifically because "we" don't know, and still don't.

> For all intensive purposes

FYI, it should be "For all intents and purposes". :)

Thank you davej, fixed. That mistake is hard-coded in me. It won't be the last time I make it.

Great Googly Moogly this place is friendly. Like Reddit with a +7 CHR and +11 INT roll.

It's great, isn't it? :) I'm just scared some day it will wear off, like the state Reddit seems to be in now...

First of all: keep the staggering to a minimum.

Second: realizing that "we should have known" and "none of this is new" isn't so much about reading news articles and being "plugged in", but rather having an understanding of how the Internet works. To oversimplify greatly, you're essentially playing a very precise game of telephone between around 10-20 different people, and usually about 1-3 different publicly-owned corporations. To be surprised at the possibility of storing packets is somewhat naive considering how simple it is to do.

Third: NSA whistleblowers have been stating what was more or less going on for years… (here's a talk by Binney from last November [0])

Funny enough, I wrote a post on this subject matter too before the Snowden leaks where I included the video as well…[1]

[0]: http://techtv.mit.edu/embeds/21783?html5=true&size=full&cust...

[1]: http://blog.pictobar.com/post/47787766458/why-so-silent

Maybe I am just having trouble seeing the point of "see I was right all along"? Why would we be upset at the newcomers to the ranks of the enlightened? I would prefer to just nod, point to the preexisting evidence, instead of driving people away with unproductive "I told you so" hostility.

I agree completely. We need more education on the subject as opposed to back patting, and we definitely don't need to attack the very people that need to hear and understand the reporting most, as the person you are replying to is doing, by calling them naive. A bit sad imho.

My issue with the conversation now that this has gone "mainstream" is that people are now allowing the media to shape their viewpoints (like everything else that seems to blow up in peoples minds who are normally distracted with reality tv or how awesome they think their life is[personal experience from family members/friends/how I lived for some time]), without digging further beyond what people are talking about at the surface.

The emotions are most likely to be anger and disgust of having their sense of reality shattered, inciting most people who feel powerless to change their habits, to go and protest. And as we all have seen around the world and even within the united states, protests can get pretty hairy, pretty quickly and not in the favor of people who want to live peacefully…

Outside of the issue of inciting the masses to act out physically, there is very little public "mainstream" acknowledgement that corporations are collecting and sharing the same types of data (and more) between one another, where issues surrounding any type of morality become selling points for products. So then the theoretical situation becomes: Government agrees to stop its dragnet programs, non governmental entities will continue to do so as long as people use their services… where's the protest for that (and when that comes they'll hire private contractors to protect them and their interests [remember OWS 2011])?

I posted this a while back on information asymmetry and the surveillance state [0], which lays out simply what is going on now in the minds of people and what is at the core of the issue people are talking about. I also propose an idea about the direction I feel would be more beneficial for the energy to be placed on my post as apposed to the logical conclusion of where all the anger will be placed by people who are now willing to enter the conversation from recent "mainstream" exposure [1].

[0]: https://news.ycombinator.com/item?id=6042241 [1]: http://blog.pictobar.com/post/52533760444/the-nsa-is-closed-...

I agree with you 110%.

That being said, I can also imagine how frustrating it must be to be a person who's spent years (maybe decades) worrying about something that's really happening, only to have their concerns dismissed with a wave of the hand or marginalized as "tinfoil hat" conspiracy theories. It's not hard to imagine how that could sour the disposition of even the sunniest person.

Welcome to /r/conspiracy - come on in!

The technical possibility isn't the new and staggering part, it's the profound lack of morality, respect for any ideal whatsoever, and compete apathy towards the oaths these people took to serve us.

They have compeley misused the power we granted them in sacred trust. We should remove it from them at once. If this has become impossible, we need to know that as soon as we can.

I could not agree with you more re: removing them at once. Sadly, I don't think an overly militarized police force, rapid transfer of wealth to the top and the post-911 power grab is going to challenged anytime soon.

Most Americans still believe they have more to lose than to gain by asserting themselves...

> The technical possibility isn't the new and staggering part, it's the profound lack of morality, respect for any ideal whatsoever, and compete apathy towards the oaths these people took to serve us.

Again, I'll chime in as the resident apologist. The people working at Fort Meade are not evil. They truly believe they're doing a great service to the nation. They may be wrong, and they've certainly thrown privacy out the window. But they are following an ideal: national security.

Post 9/11, the nation went on a war footing. We reacted the way we did to the Nazis and the Soviets. And in their search for an existential threat, the intelligence community seized on nuclear terrorism. These analysts live in constant fear of the day they miss a piece of information and New York, Washington, or London is enveloped in a mushroom cloud.

The best explanations for this type of reasoning that I have heard came from an unlikely source, my grandfather. He's a former FBI agent and WWII Navy veteran. In war time, we threw all sorts of civil, economic, and political liberties out the window to defend ourselves. When I asked him how this was allowed to happen, he said simply, "When you're facing an enemy that wants to cross over the hill into the valley where you, your family, and everyone you've ever known or loved lives, you'll do anything to protect them."

Our grandparents grew up with the threat of the Nazis. Our parents faced the prospect of annihilation by the Soviets. We have had the luxury of coming of age in a time where there is no credible threat to our very national and physical existence.

As a result, it's difficult for us to understand the mindset of someone that spends all day, every day, thinking of the most horrible ways we could be attacked, and then trying to devise countermeasures. It's almost inevitable their perspective on the balance between security and privacy is altered.

I'm not saying this reasoning is morally correct or justifiable, especially when applied to the current surveillance programs, but simply that it is understandable.

The key danger is that these efforts are qualitatively distinct from those in previous generations. The difference between extraordinary measures now and then is twofold.

First, our capacity to surveil the citizenry has exploded over the past two decades, and our legal framework is still grappling with that change. The courts are having trouble understanding that a change in scale can be a change in kind.

For example, it's one thing to have the occasional surveillance flight to search for drug operations. It's quite another to have aerostats and quadrotors watching every inch of a city all the time. But the legal rational that there is no right to privacy in public spaces allows both.

Similarly, it's one thing to say the records generated by my water company are business records not subject to the Fourth Amendment, but it's quite another to use that rationale to justify monitoring the location of my cell phone simply because my cellular provider maintains the records.

Second, wars have a point where they end, and the extraordinary measures are supposed to be reversed. That's why the "war on terror" and the "war on drugs" are so dangerous to civil liberties. They essentially extend the extraordinary measures during wartime to police problems that have no logical end.

I agree that we've gone too far as a nation. The fact that these queries don't require FISA orders flat out shocked me, even as a careful observer of these issues. But let's not demonize the individuals. After all, they're only doing what the people demanded after we were attacked. This is a democracy, and immediately after 9/11 such measures were resoundingly approved by the public and our representatives, beginning with the PATRIOT Act.

None of that changes the current reality however. We must slowly learn the lesson the British did when dealing with terrorism. If you treat it as an ordinary police matter, something that will always be present, you deprive it of its power to shock, from which it derives its effectiveness.

The fact is that the war on terror must now end. It's time for a return to normalcy.

I'm not American, so I'm wondering: was the public really actually behind the PATRIOT Act, or were they merely giving leeway in a time where everyone was supposed to go along? Or were you thinking that's the same thing?

Same with the politicians; were they really for it, or simply incredibly afraid of the political suicide that would be the results of standing up against it? Because this was a time when people did not question Bush. From today's perspective on his administration's actions, that seems odd, but it was the reality at the time.

How can anyone really be behind something they barely know anything about? When a bill like that comes around, the general reactions usually run from If You Say So to They'd Better Not Screw This Up. Some are completely deferential, some are completely skeptical. Nobody knew the details of what the law entailed for certain, so argument over it is like kickboxing on a waterbed: pointless, but vaguely resembling real fighting/debate. EDIT: to be clear, the general assumption is that Congressman know enough about the law to understand it (some things can be withheld from the public).

> incredibly afraid of the political suicide

Afraid is not the right word. Aware. When all (public) evidence concerning a law says "fight the terror!" and buildings are still blowing up, you'd have to represent a very interesting district to be "soft on terror".

The public was behind doing something. Much of Congress didn't want to be seen as impeding something.

It was obvious from the length of the act alone that even Congressional staffers couldn't have read it carefully between the time of submission and the time it passed. Quite a few people that I knew were weakly opposed, but the sunset provisions may have made it more palatable.

It takes character to stand up and defend doing nothing when something "must be done".

>It was obvious from the length of the act alone that even Congressional staffers couldn't have read it carefully between the time of submission and the time it passed.

This is a little off topic, but I always see this trotted out when people talk about big laws (like Obamacare, PATRIOT Act, etc) and it's not really true. Lawmakers usually work with and read a "normal language" version of laws that then gets transformed into a stricter legal version by staffers and experts. They will look at the actual legal version of the law if they care about a specific rule or section, but they usually don't need to.

It is an incorrect characterization when referring to the Affordable Care Act, as that went through so many revisions and debate over such a long period, that anyone who did not read it has zero excuse (including the public who allows itself to be misinformed about its contents). But it's not quite unfair wrt the PATRIOT Act. There was widespread reporting, complaining, and outright indignation that the PATRIOT Act was never read by a majority of congresspersons who voted for it. It was so massive, that there was little time to actually read the legal language overnight.

Of course, I expect my lawmakers to actually read the legal language.

The point is more that for most lawmakers there's not really a need to read all of the nitty gritty legal language. If you're a House Rep from Kansas who's core issue is corn subsidies, reading all of the PATRIOT Act isn't really going to do you much good. Instead, you read the summaries and listen to the opinion of the experts in your party who have read the whole act.

It's important too to note that this isn't a "big law" or even an American thing. Virtually all bills of any substance work this way and it's pretty much standard practice in most countries.

That being said, I'm not defending the PATRIOT Act. I just think the argument that not enough people read it is weak, especially considering all the real arguments you can make that actually attack the substance of the act.

You make some decent points. However, I'm still going to counter that 'the argument that not enough people read it'--i.e., proposed laws--is strong, not weak.

The point is that for all lawmakers, there is both a need and sworn obligation, in addition to national expectation, that they read all the nitty gritty legal language they are voting on, by which all Americans are bound to abide.

That's what lawmakers are there for--to know what in the hell they are passing as laws. If they can't be bothered to do their job--which, at the national level, goes far beyond just securing corn subsidies, because they're voting on legislation that touches on all Americans--then fuck 'em. Throw the bastards out on their asses, and send them back to the cornfields.

For the most part, we as Americans didn't actually ever read the Patriot Act, and we didn't get to vote on it. Our representatives that we elected before we ever knew 9/11 would happen voted for it in a climate that made it politically suicidal to not vote for it.

To be clear, the "hawk" politicians (and let's be honest, -many on the left) believed in the legislation but also exploited the tragedy to ram it through and neutered the ability of the other side to have a reasoned debate.

Our population was attacked, angry, and for the most part followed the lead of politicians who said we needed these laws to fight the people that attacked us.

In the aftermath, the scrutiny on the part of the American people never materialized. You're basically witnessing the moment where the most scrutiny on these types of programs/laws has ever occurred since 9/11. Worth keeping in mind that many components of these surveillance programs also predate 9/11.

Many of us were, and still are, against it. Its passage was very questionable and suspicious, particularly regarding the lack of informed and reasonable debate on its requirements and broadly invasive permissions. It was passed overnight. There was word that many (most?) congresspersons did not even read the bill before passing it by a huge majority. It was emotionally charged and rational criticism was nearly non-existent before it was passed.

Only 66 Representatives voted against it--62 Democrats, 3 Republicans, 1 Independent. Only 1 Democratic Senator voted against it, while another Democrat abstained from the vote.

At the time the Act passed, Americans were in the midst of a fear frenzy. It was a pervasive culture of fear and panic, the likes of which I can only compare to anti-Soviet fears of the Cold War. People all over the country actually went to stores to buy all kinds of emergency and survival supplies to build up their own anti-terror kits (I forget the name for this that was popular at the time).

Many of us questioned Bush from the moment he was declared the winner of the 2000 election by the Supreme Court. We took part in protests all over the country after 9/11 to oppose the buildup to war in Iraq. I took part in protests in D.C. It was all ineffectual. Fear gripped the country and few paused to consider the long-term ramifications of the actions taken in September's wake.

Yes, they really were behind it here in America. The overwhelming majority of the people on this planet are unamendably stupid.

While I feel that the programs the NSA employs are profound existential threats to our liberty and rights, I do agree with you on the balance that the human parts that make up the whole of these organizations fundamentally see themselves as benign and beneficial on the balance. I think it bears mentioning, and its worthwhile to keep this in mind while we do the necessary work of attempting to dismantle and remove a lot of their power and tools, -the ones that have gone far past the line.

Demonizing people and falsely assigning ill-intent doesn't help us address and correct the problem, even if it feels good to do so. I personally have to fight the urge constantly myself because I feel so strongly in the immorality of the net output of the programs themselves.

The issue is that we need to demonize the people who are in fact evil and deliberately built this out and got it going. That list is surprisingly short:

GHW Bush

GW Bush

D Cheney

D Rumsfeld

C Rice

G Clapper

G Alexander

P Wolfowitz

These are the guys that created the orders that the soldiers are following, and the war they are dying in for these criminal's profits.

Naive and unnecessary. The Patriot Act was overwhelmingly supported across the aisle. And it should be obvious by now that Obama is an enthusiastic supporter, based on his treatment of Snowden. Not to mention Pelosi and Feinstein aggressively defending the government's right to suppress information. This has nothing to do with party affiliation. If you believe in Republican Vs Democrat, you're still in the Matrix, and, sadly, sipping the koolaid.

Blue pill for me! What a lovely color; it matches this donkey on my t-shirt!

I think you're missing what I am saying, which firstly, is in no way party related.

The people I listed have a decades long history which brought them to the US Coup of 9/11: Cheney in particular.

The above are at the core of PNAC, the CIAs takeover of the executive branch (both Clinton and Obama are their puppets here)

GHW Bush has been running shit since the 70s.

Cheney setup the framework for the current MIC exploitation of the world when he was in Sec. Defense position in the early 90s - then setup Halliburton to be in the position to receive all the mandated private-sector contracts so the military could focus on its "core" -- the same with the Carlyle group.

(Carlyle owned CRG West (MAE WEST) and other fiber infra and DCs)

These guys worked diligently to put all this into place. Obama is just a puppet who was meant to quell the outrage that the Bush regime was bringing.

I posted a list of the key players in this, I did not post any party affiliation....

I can provide a hell of a lot more detail than this too - going back to 1920 with these guys...

It is excessively naive and completely discredits your otherwise potentially salient points to suggest President Obama is a puppet.

You're wading far too deeply into conspiracy territory to suggest that this puppet 'was meant to quell' anything. He is a leader whose administration stands and falls on its own merits.

I think it is naive to believe that each and every administration "stands and falls on its own merits" -- and then in the same breath talk about partisanship.

There is no party but the MIC party - and clearly, the NSA owns that party.

America has died, completely, 100%. There is no such thing as "Land of the Free, Home of the Brave"

This is tin-foil hat territory. The CIA was practically dismantled under Bush 43, and the intelligence agencies fight amongst one another like boisterous stepbrothers. To think the intelligence agencies control the government is vastly overestimating their internal political cohesion and capability.

The IC isn't running the government. They've got their hands full just running themselves.

The idea that we are not free is absurd. If I want to hold a rally for the Ku Klux Klan, that activity will be protected by the full force and power of the United States government. I can worship as I wish, read the books I choose, and write whatever I want (excepting direct threats of violence) with little fear, knowing that laws and courts stand ready to vindicate my rights.

I would take our extensive package of rights over single party political control, strongman leadership, civil law jurisdictions, and common law libel standards any day.

We are certainly no longer the most free nation on the planet, which saddens me deeply. But we are certainly amongst the best on that metric.

You forgot to add President Obama and other current leaders to the list. Expansion and utilization of these programs has also occurred during his administration.

You're clearly being partisan.

I was talking specifically about the ones who setup the current situation. Clearly there is no argument that its been embraced and extended by the current puppet regime.

That's way too simple. Many people on that list belong on that list, but...

The American people overwhelmingly approved the Patriot Act, and the idea of surveillance, and the war on terror, and the actual wars on place.

The Obama administration resumed surveillance programs which had been previously shut down.

The military industrial complex has been growing steadily larger since the 1950s.

Congress people from both parties repeatedly approve the growth of the defense budget, and especially parts which gain them money and jobs for their own states and districts.

There are certainly people to demonize, but sorting them out from the well intentioned would be incredibly complicated.

> The people working at Fort Meade are not evil. They truly believe they're doing a great service ...

That isn't really a strong argument. Firstly, their actions is supposed to reflect the ideology of the US citizens in general. If it doesn't, either they are not being administered as well as they should be or they are purposefully ignoring the will of the citizens. Secondly, the idea that because they truly believe that they are doing great service doesn't actually justify any of the actions. If we are forgoing the label of evil because they think that they are doing great work (and I am OK will that, I hate the label 'evil'. It is unconditionally partisan) then it does question whether Nazis/Soviet union deserved the label as well. Because I fear that they too believed in their actions.

> our legal framework is still grappling with that change

US legal framework does not seem to be struggling (I am not a native speaker, so I am assuming that is what you meant). It has expanded the power to monitor and interfere knowingly and willfully. Let's not blame this on misunderstanding or incompetence. While it is the first thing that this should attribute to, the people who have built this system seem highly skillful and knowledgeable. If you claim that decision makers do not understand the new world that has suddenly bubbled up, well it's your responsibility and that of the NSA employees who seem to be following orders without questioning, to either make them understand or replace them. And in all fairness, US voters did. The man even won a Nobel Peace Prize for some reason I cannot understand. But his actions behind the doors seem totally contrary to what his words have been in past. Not really the fault of the voters but it definitely raises questions if he truly understood the costs and still took the leap.

[Edit: grammar]

> Firstly, their belief is supposed to reflect the ideology of the US citizens in general. If it doesn't, either they are not being administered as well as they should be or they are purposefully.

I think this is a very difficult question to answer. If you're a lowly NSA tech tasked with something seemingly mundane (say, writing some automated tool to be used by an internal billing dept), at what point do you refuse to contribute to an organization that may be operating against the will of the people? Who is responsible?

Concentration camps? Me!? No! I just load the trains. It's a tough job market and I need the work. I'm just a regular Johan just trying to get by.

> The people working at Fort Meade are not evil. They truly believe they're doing a great service to the nation.

Evil doesn't require intent. Some of the most evil acts in history were carried out by people who believed they were doing a good and moral thing. Most evil people don't go around thinking "I'm going to be so evil today!"

I suspect you are correct and that the vast majority of NSA employees think they are doing the right thing for America. That doesn't make their actions any less evil.

>The people working at Fort Meade are not evil. They truly believe they're doing a great service to the nation.

I don't want to Godwin the discussion here, but it's not at all rare for people to act in an evil (or whatever you want to call it -- bad, harmful) way while not recognizing their own actions as evil.

That people don't think their actions are evil doesn't prove that their actions aren't evil.

Add to that, evil acts are almost always done in service of an ideal. For example the USA has economically and socially gutted many nations by force in service of the democratic/free-market ideal. Yet it's rare to find an American who sees it this way. US-USSR proxy wars in the Middle East and Latin America from the 60s-90s weren't destructive, we were just trying to help those countries out. We wanted to modernize them, to improve their lives, not to destroy them. They were just too uncivilized, too barbaric to get it. Why would they hate us for that?

Hence 'ideology'. Easy to serve, hard to view objectively when you've spent a lifetime on the inside.

>We have had the luxury of coming of age in a time where there is no credible threat to our very national and physical existence.

The Berlin wall didn't fall until 1989. The Soviet Union didn't dissolve until 1991. The period of 1991-2001 was spent fighting proxy wars in former USSR terrories or allies [1]. Iraq. The Yugoslav Wars in Bosnia, Macedonia, Kosovo. Haiti. All of this was an extension of the cold war. The Red threat didn't officially end until 09/11/01, Communism continued to be a spectre held over the head of the American public. It's just the discourse shifted from "the USSR has bombs that can kill us right now" to "Communism is bad therefore we're preventing it from spreading". The constancy of threat and surety of the potential for complete annihilation was always there.

And of course, from 2001 on everyone spent all day, every day thinking of the most horrible ways they could be attacked by terrorists. With great encouragement by media and government apparatuses.

>But let's not demonize the individuals. After all, they're only doing what the people demanded after we were attacked.

Again avoiding Godwinning, but to a certain extent you must demonize the individuals. Else there is no incentive for people to be vigilant of runaway ideology, like the US is operating under currently. Else there is no incentive for individuals to formulate a moral compass external to the state, because why bother when "they told me to do it" is a legitimate excuse? The state idology becomes your morality. After all, you're just tryin' to put food on your family.

[1] - https://en.wikipedia.org/wiki/Timeline_of_United_States_mili...

> That people don't think their actions are evil doesn't prove that their actions aren't evil.

Certainly not. The issue is not their beliefs, but rather the reasoning behind them. Different experiences of the world give rise to different world views. The world view of those that operate, condone, and approve the surveillance arises from a set of historical understandings and modern experiences that neither you nor I share.

To suggest that the scare tactics of CNN and the like is comparable to the psychological effect upon an ordinary analyst of regular intelligence reports of weapons-grade uranium being smuggled out of Russia via Kazakhstan is naive at best.

The threat of true national annihilation, not a specter concocted by a manipulative elite, has been the norm rather than the exception throughout history.

Modern totalitarianism has its roots in a not too distant past in which totalitarianism was the surest defense against large armed groups of humans that would burn your fields, kill your family, and subjugate your people.

That threat didn't disappear until very recent times. The cultural history of the American people is replete with threats to our existence: the CCCP and Warsaw Pact, the Axis, the German Empire, Spanish colonial North American empires, the British Empire, the Quadruple Alliance, the Normans. The intelligence community takes it's cues from a long history of existential threats.

What seems so obvious to us is that the current world is stable, and thus extraordinary measures to protect our safety aren't justified. Those charged with national security take a longer view. They see our nation as balanced on a knife's edge between internal strife and external threats. And thus, threats to either must be vigilant observed, documented, and understood, so that if the time should come when a conflict does occur, we stand prepared.

That line of reasoning is often alien to privacy advocates. I neither endorse it nor deny it. I simply acknowledge that those who study, train, and practice for our defense are not naive when it comes to the risk of violating civilian privacy. They simply set a different value to each of the variables in the risk-reward equation. You may disagree with those values, but it is important to understand them. Blindly denouncing such views as morally bankrupt simply factually incorrect.

> The Berlin wall didn't fall until 1989. The Soviet Union didn't dissolve until 1991. The period of 1991-2001 was spent fighting proxy wars in former USSR terrories or allies [1]. Iraq. The Yugoslav Wars in Bosnia, Macedonia, Kosovo. Haiti. All of this was an extension of the cold war.

The wars you cited were in no way related to the Cold War. Yugoslavia was a strategically unimportant area, relevant to no one in the geopolitical sphere.

The intervention occurred as a direct result of ethnic cleansing that was taking place in obvious, organized, and deliberate fashion. To suggest otherwise is simply incorrect. I've spoken with the head of UNPROFOR from the Srebrenica Massacre. It was a war crime on par with the worst parts of World War II. Clinton himself stated that his reluctance to intervene was based upon the "ancient ethnic hatreds" argument of Balkan Ghosts. The Yugoslavian intervention was about genocide. As a simple fact, it had nothing to do with the Cold War.

> Communism continued to be a spectre held over the head of the American public. It's just the discourse shifted from "the USSR has bombs that can kill us right now" to "Communism is bad therefore we're preventing it from spreading".

Containment of communism was simply not a factor during the nineties. Moscow was crushed, the former Soviet block in shambles, and Russian interests retreating from throughout the world. Hence the remarkable cooperation on nuclear arms, energy policy, and democratization between the Yeltsin administration and the Clinton administration.

>I don't want to Godwin the discussion here... Again avoiding Godwinning...

I believe the Romans had a term for emphasis by pretended omission.

> to a certain extent you must demonize the individuals. Else there is no incentive for people to be vigilant of runaway ideology, like the US is operating under currently. Else there is no incentive for individuals to formulate a moral compass external to the state, because why bother when "they told me to do it" is a legitimate excuse? The state idology becomes your morality. After all, you're just tryin' to put food on your family.

In a totalitarian state, this argument would indeed hold water. However, you gloss over the most significant part of the counterargument. We didn't simply allow extraordinary efforts against terrorism, the people of the United States overwhelming endorsed it.

A democracy is beholden to its people. Its morality is, by definition, derived from the consent of the governed as expressed through the democratic process. Vox populi, vox dei, as it were. To point fingers at talented and intelligent programmers, people with whom we would be excellent allies and friends in other circumstances, excuses the true culprits: us.

We are to blame for this leviathan. Not the NSA, not Obama, not Bush, not the DNI, DIA, CIA, FBI, or any other amorphous acronym.

We need to understand the reasoning of the those that built these programs, not simply dismiss them as callous power hungry sociopaths. We need to grasp the history that informed their reasoning, both recent and that which began far before that day in September.

Most importantly, we need to remember that blaming individuals does nothing to prevent the true failure, a systematic disregard for the right to privacy and the guarantees thereof provided by the Constitution.

The war is perpetual, citizen. Remember:

"War is peace. Freedom is slavery. Ignorance is strength."

Orwell in 1984: "Part of the reason for this was that in the past no government had the power to keep its citizens under constant surveillance. The invention of print, however, made it easier to manipulate public opinion, and the film and the radio carried the process further. With the development of television, and the technical advance which made it possible to receive and transmit simultaneously on the same instrument, private life came to an end. Every citizen, or at least every citizen important enough to be worth watching, could be kept for twenty four hours a day under the eyes of the police and in the sound of official propaganda.")

It's a surprisingly interesting novel.

Yes, I quite agree.

I have a tremendous amount of respect for those in the security services, who have been given a rather difficult job to do, and who seem (from the vanishingly small amount that I know) to be approaching it in a professional and objective manner.

I have no desire to be nasty, and if I have personally offended anybody by what I have written, I most profoundly apologize for the hurt.

However.

This is an important issue, and it deserves public attention and a detailed debate. I hope that some of my provocative wailing and doom-mongering has done what was intended: provoked some thought and consideration.

This is, after all, politics, and, as I have mentioned before, we sometimes need to make a caricature out of our own positions in order to make a point. Omlettes and eggs and all that.

So concentration camps were understandable???

You are nuts if you think that that was acceptable given the circumstances.

Just doing my job is not sufficient in jobs such as these.

You're confusing understandable with permissible.

I understand Nazi concentration camps. It was a manipulation of nationalist sentiment against an imagined internal enemy, conveniently one that could be dispossessed of a great deal of property, coupled with a never before seen combination of the pure survivalist id meeting modern state capitalism.

I understand United States concentration camps. While we certainly didn't starve, gas, or force Japanese, German, and Italian Americans, we did relocate large numbers of them to temporary camp facilities for the duration of the war. It was believed that recent immigrants and their children might harbor loyalty to extremely dangerous enemies and could serve as a fifth column in the event of an invasion. For what it's worth, despite the indignity and suspect constitutionality, that's a far cry better than most nations have acted in similar circumstances.

Both of those events are understandable, in that I can understand the thinking of the people involved. It does not mean I morally condone it. What I'm attempting to combat is the notion that all acts with which one disagrees must be the result of moral bankruptcy or internal failing.

Usually there is a logic, however skewed, behind even the most heinous events in human history. The first step to preventing those events is to understand that logic. Only then can we address the root causes of the problems we wish to solve.

In this case, I'm suggesting that the root cause was a panicked citizenry seeking shelter from a very real threat, not a government seeking to blindly expand its power. That's an unpopular opinion, but alternative interpretations lead to different actions.

Interestingly that's not the part I find new or staggering at all. I suppose that's just an exceptionally cynical worldview at work? No matter how "sacred" the trust I always expect this amount of power to be misused to this degree when it's secret and consistent with the ideologies present among those with that power.

>Second: realizing that "we should have known" and "none of this is new" isn't so much about reading news articles and being "plugged in", but rather having an understanding of how the Internet works.

These are exactly the kinds of comments I'm talking about. The preponderance of people affected by this program on the globe (a staggering amount if you will) had no knowledge of this because the media failed, and are not, in fact, technically savvy on any level and don't understand, at all how the internet works in relation to the technologies employed by these programs.

>To be surprised at the possibility of storing packets is somewhat naive considering how simple it is to do.

For the vast majority of the potential consumers of this knowledge, this just simply is not the case. At all. They aren't being naive. This is highly technical to them and severely under-reported, and where it was reported it was not explained terribly well, nor was there meaningful conversation surrounding the reporting's aftermath.

But congratulations, rmrfrmrf, on being one of the select few that are not naive. We need to get you some sort of prize.

> These are exactly the kinds of comments I'm talking about. The preponderance of people affected by this program on the globe (a staggering amount if you will) had no knowledge of this because the media failed, and are not, in fact, technically savvy on any level and don't understand, at all how the internet works in relation to the technologies employed by these programs.

Of course at least the mainstream media (MSM) failed. Why? It's a very old story, rock solid in the media: An MSM media company is in business to make money. They have some old techniques for doing so. Their main technique is to get eyeballs for ad revenue; for that their main technique is to grab people by the heart, gut, and below the belt, always below the shoulders, never between the ears; the content is essentially only light entertainment following the framework of the ancient Greeks we now call formula fiction; the content is nearly never the information needed by an "informed citizenry".

The best hope for the information citizens need is Web sites on the Internet and search engines that can help people find that information.

I understand your point, but you fail to realize that comments like "why are you surprised?" induce a kind of digital bystander effect: they're essentially defusing moral outrage via social proof. If you read a comment like that, you may think to yourself, "well, this originally seemed like something worth loudly protesting, but if everyone already knows about it, then I guess it must not be that big of a deal." It has the effect of numbing outrage regardless of the outrage's merit, and I can't see how that's productive.

If you feel that the outrage is in fact without merit, then attack that on logical/rational grounds, not by appealing to social proof.

To paraphrase (or is it a direct quote?) Jacob Applebaum, "Quiet, please. Adults are talking."

This is staggering, and to chide others for being staggered is the worst kind of truculence.

More relevant, and useful: What are we going to collectively do about it now that we know, beyond a doubt, what exactly is happening?

You're both right. If you read the PATRIOT Act, it's easy to look backwards and see that the things we're becoming aware of now are logical extensions of what was being asked for way back then.

It is, however, VERY easy not to have been able to have that foresight, and I think that the insights people were expecting the government to have been constrained by the fact that all the information of value is collected by neutral third parties. Google, Yahoo, Twitter, etc., aren't likely colluders with the government.

Plus, at the time of the PATRIOT Act's passage, there wasn't quite as much information being put on social media, or out to the public in general. Not as much was online, digital, or otherwise easily indexable.

There were those predicting this sort of possibility before the PATRIOT Act's enactment, and since, to be sure, but you shouldn't feel responsible for not having seen the signs yourself, or for having heeded the words of what probably seemed like kooky overreactionaries from back in the day.

The funniest part about this, to me, is that somewhere, very quietly, Richard Stallman is quietly telling us all the he told us so, and he's absolutely right, and always has been. Neverminding that, he's largely seen as a crazy old paranoiac who we should respect for his IT knowledge, while having to forgive the rest of his eccentricities.

If Richard Stallman is quietly berating us somewhere, he can go fuck himself. Part of educating the masses is being a person who people want to listen to. If he failed at that, he's no better than anyone else, and perhaps far worse, because of all the lost potential.

Part of being intelligent and shrewd is listening to the words that people say, and judging arguments based on their merit. The idea that Stallman should go fuck himself for not dumbing down or tarting up the message enough for you to pay attention to him makes you the asshole, not him.

In my experience, telling people to do something hard (open source, keep privacy, etc.) in the face of a barely perceived danger (government is coming to get you) is kind of a hard message to get heard.

Aside from that, I didn't mean to seriously suggest that he's out there passing judgement on us so much as I was attempting to acknowledge how hypocritical we are for having disregarded his message because of his eccentricities. I think your statement, that he should actively try to be more popular for us to care, is further proof of how wrong we are to be that way.

In an ideal world, your response would have made a perfect satire of how Americans are likely to react in the face of the responsible elder telling us to eat our proverbial vegetables. That is isn't saddens me.

So you're expecting the world to come to terms with Stallman, rather than the other way around.

Think about that for a minute, and then explain to me why that makes more sense.

Speak for yourself. Stallman is a massively influential thinker that has indisputably changed the world positively. A lot of the world has reshaped itself to attempt to resemble Stallman's dreams. His contribution was to have the dreams and to share them in material ways, and he didn't even owe us that.

You're in the bizarre position of criticizing him for being right. You're expecting Stallman to figure out a way to market to you, rather than expecting yourself to figure out how to evaluate arguments and evidence rationally. Think about that for a minute, and then explain to me why that wouldn't make more sense.

You seem to think I'm the one who has a problem with him. I think he's always been dead on, and don't disagree with you in the slightest about his vision.

Where our expectations start to misalign is the part where he's been ignored because he doesn't know how to be a consummate human being (let alone marketer), and you say it's everyone else's fault. Idealism is fucking useless.

Be careful what you wish for because you might end up with Ted Kaczynski.

> I'm a practically addicted news junkie ...

More like a news sheep. The mass market news is and has always been 49% fluff and 49% lies.

Comments from people who already knew what the NSA does are not "self aggrandizing". The are other-insulting. You should rightly be ashamed that you walk through life in a news fog of up-to-the-minute minutiae. Read books by retired insiders, talk to current insiders and contractors. That's the only way you will learn anything about anything. To wait for the newsmen to do it for you is to sign your mind over to tampon salesmen.

The NSA story is staggeringly unimportant. Every government, many companies, and rather a lot of organized criminals run intel and counterintel operations. It is just a fact of life, like antibiotics and highway construction. It is inevitable that there must be a national American signals intelligence organization.

What os staggetingly important is why the NSA alone, out of all the spy organizations, is being singled out for a comprehensive media war. The most likely explanation is that the Democratic Party needed something to distract from its pecadillos. The next most likely explanation is that a foreign government is getting themselves some payback. In any event, if you care about this non-news, you are just another mindless pawn.

Wow, what a staggering misinterpretation of what the parent said.

Every time I post the truth about this NSA fiasco, I get:

1. Downvoted to oblivion by a hivemind, and

2. Somebody like you chimes in with a content-free emotional outburst.

So exactly what did I misunderstand?

The incontrovertible fact that this really isn't news?

The fact that every history and exposé on the NSA has been saying this for decades?

The fact that the NSA tried cramming the Clipper chip and key length restrictions down our throats to make domestic spying easier? For half a decade this was a weekly running joke on Slashdot that you had to have been living under a rock to miss.

That the previous commenter claimed to be a "news junky" and then admitted that by news he means the mass media—a pack of tampon salesmen and political hatchetmen.

>That the previous commenter claimed to be a "news junky" and then admitted that by news he means the mass media

Absolutely nowhere did I say, or even begin to imply that. In fact, I explicitly called out the mainstream media for being complicit and/or not reporting on this issue while indicating that much of what is being reported was already known to me. Not only did I NOT say that I get my news from the mainstream media, the implication was, if anything, that I did not. The mainstream media is about the last place I'd look for competent coverage of this issue.

You're terrible at reading comprehension. Terrible. You make a lot of assumptions, all of them wrong, then proceed to insult other people based off your incorrect assumptions.

Additionally, the only thing incontrovertible is that this is news to the vast, vast majority of people who are affected by these programs. Those are the real numbers. But I know you. You're part of the Pedestal Crowd furiously patting themselves on the back. Good for you Danny. Atta boy.

Yikes, please read the parent's response to yours. He explains better than I can how you completely misinterpreted his initial comment.

I think it's just a demonstration of complacency more than any actual knowledge on the subject. I've noticed it's invariably my non technical acquaintances who are the first to pontificate on how this is all somehow boring old hat.

> It has become a bit of a pet peeve of mine recently to see self-aggrandizing comments from users around the net about how "we should have known" and "none of this is new."

I agree that "know" is a bit too glorifying. I propose "suspected".

I don't find this surprising at all. Practically 99.99% of a normal user's Internet activity is centered on Facebook, Google (including Gmail) and a handful of other sites. The amount of data everyone is requiring in order to provide a service also includes pretty much anything you need in order to track someone.

It's not news you need to pay attention to but some of the more theoretical aspects of networking in a second-year course.

I have nothing wrong with people having suspected it for a long time, or even saying so. I suspected it for a long time as well. My problem is with the attitude many people seem to have once evidence confirming those suspicions comes out and they go on about how the evidence means nothing because they knew it all along. No, the evidence confirms their suspicions, which makes it incredibly important!

Ultimately, whether they intend to or not, such statements end up making other people who are hearing about this for the first time more complacent about it because they come into the comments and see a bunch of people going on about how it's nothing new and therefore the new information is no big deal.

I share your pet peeve and I can only assume that the "meh, no big surprise here" response stems from two things: wanting to sound just as knowledgeable as the person who brings up the topic (despite not having any new information); and at the same time justifying their complacency about the issue.

Exactly. That type of cynicism and one-upmanship is grating.

>and at the same time justifying their complacency about the issue.

A good theory, as I have an extremely difficult time imagining anyone in an activist (non-complacent) stance on this issue ever reacting like that to these revelations.

From the slides, apparently a node in the system just connects at an ISP or peering site and grabs all the packets. Then they essentially 'parse' the packets to TCP/IP sessions, logical user sessions, e-mail messages, etc.

Then back at HQ, can send the node what are essentially 'filters' to return 'alerts' and the associated content.

So, point: As a system, it's quite obvious. As software, it's quite routine.

And, from their description of working with anomalies, they are being just intuitive and elementary and not at all advanced or powerful.

It would appear that a terrorist Internet user could do fairly well beating that system by using a proxy server also used by many other Internet users and also using a lot of strong encryption -- PGP used well might be strong enough.

See? No "direct access!" Google/FB/Apple's statements, totally reassuring.

For me, personally, it's not about "look how smart I am" as it is genuine surprise that the story actually seems to be sticking this time.

I'm glad that people are paying attention, but especially early on, it wasn't entirely clear that Snowden's leaks were substantially different from the leaks that have been coming out of the NSA for years that never got traction in the media.

I think the type of leak is substantially different, the other leaks were all somewhat hard to describe. The Snowden leaks have the names of well known companies in big menacing letters.

The traditional media is complicit. And it isn't some grand conspiracy either, they just share the same interests as the rest of the establishment, being part of (and/or owned by) the establishment themselves.

There is good independent media that has been covering the story for years though. Here's a Democracy Now story from February 2005:

http://www.democracynow.org/2005/2/10/no_place_to_hide_award...

Democracy Now has an incredible archive on this subject too. Right now it starts here:

http://www.democracynow.org/topics/nsa/7

Specifically, they've done some great interviews with previous whistleblowers:

http://www.democracynow.org/appearances/william_binney http://www.democracynow.org/appearances/russell_tice http://www.democracynow.org/appearances/thomas_drake http://www.democracynow.org/appearances/jesselyn_radack

Other interesting guests:

http://www.democracynow.org/appearances/jacob_appelbaum http://www.democracynow.org/appearances/laura_poitras http://www.democracynow.org/appearances/james_bamford

Only when Jacob Appelbaum was at Wikileaks, they also allegedly stole people's data (through Tor networks). Assange has admitted it - http://www.theepochtimes.com/n2/world/wikileaks-intercepted-...

How is that relevant to the NSA story exactly? Are you saying that the Government vacuuming up any and all data it can, and granting internal and external analysts easy access to that data, is comparable to the owner of a private server analyzing the network traffic of their servers and networks? If you want to hold private server and network operators to a standard that restricts them from doing that you're going to have a bad time.

And the purpose of Tor might be different than you imagine:

https://www.torproject.org/docs/faq.html.en#WhatIsTor

Or are you just trying to discredit Appelbaum, Assange, and/or Wikileaks?

Also, is that your article? Should you disclose that? And is there any reason you linked to it rather than the original New Yorker article here:

http://www.newyorker.com/reporting/2010/06/07/100607fa_fact_...

Is it self-aggrandizing? I suppose I'm one of those people.

I was shocked by having this laid out as well but I really did just assume this was probably going on. It was technically possible, it was politically possible and it was financially possible. If I shared the worldview of the people doing this and been in the position to do this, I would have been itching to start this level of collection and data mining.

I will admit to part of it being satisfaction at no longer getting the "oh put your tinfoil hat away, no one would do that" response whenever it came up, which was always based solely on the old "I don't like the implications of this being true therefore it can't be" argument. It's also relief that there is finally a discussion about a subject that was previously only seriously discussed by a small number of people.

I take your point that the I-told-you-so gloating isn't helpful and doesn't reflect well on those who do it but I disagree that that was ever meant to discourage discussion, if anything it was anger at the fact this discussion has taken so long to occur.

I'm betting tonight on the news, this story will be eerily missing.

Room 641A?

I've been hearing about the NSA's massive data center in Utah for well over a year, from public news sources. They have always suspected that it's main purpose was the warehousing of American's private communications.

I'm one of those "none of this is new" types. The fact is, we ALL very much should have known. Do the words "Echelon" and "Total Information Awareness" ring any bells? These were terms being used pre-9/11. There is no excuse for someone technological and with a small inkling of understanding of human nature to not have seen all of this coming. There really isn't.

If you're waiting for someone like Snowden to come along and spoon-feed you all the ways the government can screw you, you're doing things completely wrong. Oversight requires foresight.

Steve gibson has been covering how they can get at this stuff and what alternatives you have on the security now podcast.

https://www.grc.com/securitynow.htm

Briefly summarized, the only way to do secure mail is pgp, the only way to do secure chat is to avoid all the main chat networks. And microsoft actively designs their systems to be easier to access for the NSA (far beyond their legal obligation) so you may assume that any microsoft product is a direct line to the NSA.

Keep in mind also timeframes. Facebook HTTPS use -- and more so use by default -- is more recent. Remember the whole "sheep" debacle?

Even Gmail HTTPS use is somewhat recent and not original to the product.

Further, one might combine this with reporting about initiatives to gain company SSL/TLS private keys, account passwords, and the like, in some interesting speculation -- if speculation it remains.

Amongst all the rest, I would point readers towards browser fingerprinting. It's difficult for me to imagine they are not using it.

If the public is going to have some degree of counter-measures, this will include browser and other client software becoming more pro-active about anonymizing its own profile / usage profile. For one thing, stop sending highly unique fingerprint data such as font listings to every Tom, Dick, and Harry. Just one thing amongst many...

> but how are they getting all Facebook private messages and Gmail?

It was reported earlier that the NSA has installed hardware at their "partner" companies. As you certainly remember from the slides, they are: Facebook, Google, Microsoft/Skype, AOL, Paltalk, ...

What's really sickening is that you can tell that programmers or very technical people were involved at some level to design these systems which help people construct rubber-stamp plausible deniability. Whoever these people were knew full well that they were architecting systems that skirt the letter of the law if not outright flaunt it.

Somewhere there is an architectural diagram of these systems that describes how to make people check checkboxes before releasing information. CYA-oriented programming that has clearly driven the entire design of this thing.

> Users of PGP/encryption products being singled out is terrifying.

I didn't see that in the article. Do you have a citation?

It is in the slides the article links to: http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

Slide 16: "Show me all PGP usage in Iran"

Your faith in Facebook and Google not giving the NSA open access...is a bit strange.

Once again, whilst the shrill cries of protest claim that the government has gone too far in it's intrusive surveillance, the pragmatic amongst us are forced to admit that this is a capability that the state simply will not give up, even in the face of massive public protest and discontent.

Moreover, the technological trend is clear; and the avenues for sharing intimate personal information proliferate and multiply with every passing month. The debate therefore needs to shift. The question cannot be over whether the state should have access to this information. We are powerless to push on that point.

The question has to be this: Given that our state (and others) will necessarily know the most intimate details of our lives, how do we want it to behave? How do we want this information to be used? What do we want the newly intimate relationship between individual and state to look and feel like? It may well be that we come to a startling different conclusion than our initial starting points might presuppose.

There are tremendous social benefits to be had by using this treasure-trove of information wisely, just as there are tremendous dangers to be risked by using this trove with carelessness or malicious intent. However, we need to think very carefully about how we manage the relationship between individual and state; how we manage the relationship between individual and peer; and how we manage the relationship between individual and technology.

I feel strongly that this is the most important debate of our generation; perhaps the most important debate to be had in this new millennium.

I'm getting seriously irritated at the "I have nothing to hide" crowd. For starters, here are a few ways this can go horribly wrong:

* Industrial espionage -- it's big business, and I'm sure it pays better than being an NSA analyst.

* Foreign espionage -- since this gives unlimited querying power to every agent, a single "turned" agent could inflict massive damage on U.S. government and industry interests on behalf of a foreign power. The potential for double agents is huge.

* False positives and guilt by association -- being flagged as a "person of interest" and then essentially persecuted because you have fringe ideological interests, are looking up a lot of info on terrorism for a book project, have a friend who knows radical Muslims, etc.

* Corrupt use in political campaigns by incumbent politicians with access -- obvious.

* Blackmail and other corruption.

* Use by government agencies with access to spy on other agencies.

... I'm sure creative people can think of more.

This. I can't seem to make people around me understand this point.

Do you list these negatives when having that conversation? Do they not see the possibility of something like this being used against them or family members when you lay out the possibilities specifically (blackmail and industrial espionage could hit anybody)?

The "nothing to hide" trope seems to me to be entirely based on a false dichotomy that contrasts "nothing to hide" with "unpatriotic/criminal". I think this is primarily because people lack the imagination to consider the other seedier and more lucrative uses of surveillance.

If they were confronted with these other possibilities, would your acquaintances change their thinking? Or do these other risks--for example, the risk of having an employer targeted by competitors unfairly (potentially leading to layoffs), or the risk of having a representative vote against the interests of his or her district because of blackmail (potentially leading to a loss of government services and investment)--simply not resonate?

When I make this argument the most common response is that they have faith in the goodness of people and dont consider these risks to be very significant.

"Industrial espionage--"

According to a New Zealand whistleblower back in the 90s, this was one of the main purposes of the Echelon network. Imagine what happens when your larger competitor gets in bed with the NSA. According to whistleblower Russ Tice, the Bush NSA was able to request intercepts on Senator Obama, so there certainly could be enough corruption for back room deals to occur for your startup's private information.

This is yet another reason to encrypt your git traffic.

Also, how about: just because I have nothing to hide doesn't mean that I don't mind being watched.

I completely agree. I got very frustrated talking to two intelligent people (one a lawyer) about this the other day. I think everyone should be educated to make sure they know about these points so that they can intelligently explain them to the "I've got nothing to hide" crowd. The 4th Amendment and the right to privacy just doesn't seem to carry much weight with the average person any longer (if it ever did).

I think that the 'nothing to hide' crowd don't understand that the concept of things to hide could grow and grow unless the surveillance power are kept in check. Today, only terrorists have 'something to hide' tomorrow it could be activists, then journalists and so on, until they 'nothing to hide' people become the victims. http://rt.com/usa/justice-department-admits-spying-228/

People need to look at this long term and realise that abuses of power will continue to intrude larger and larger sets of the population unless they are stopped now.

Thank you for making that post. I've been repeating these points on broken-record for the last few months.

I would also like to add /family/ as a huge pressure point, akin to your guilt by association.

IOW, one may have nothing to hide, but their family member does and one can be controlled by threats to that family member. It's disgusting and insidious, but that's how bad, scared and dangerous people operate. We are not using our intelligence if we allow ourselves to be vulnerable in this way.

Your points are excellent. And even if we all do have nothing to hide, an agency of the government that decides on its own to break the 4th and 1st amendments, and reinterpret the law practically out of existence, in secret, and lies about it to Congress in its oversight capacity, is A Bad Thing(tm).

Anytime an adversary (opponent in a lawsuit, police investigator, promotion candidate, political candidate, etc) is owed a favor by a member of the intelligence community, you are going to lose.

The whole of your life may be innocent, but a single interaction (searched for porn? vented about someone in a private conversation?) taken out of context can almost certainly destroy you.

Api, please make a post expanding on these ideas. Then post it here and send some copies to international news sites, like the guardian.

We need to wake up, your post would help a lot.

I'm having the same problem as some other commenters here in that people don't seem to care about these hypothetical situations. They mostly trust the gov't to only track the "bad guys". Are there any good cases of the above-mentioned actually happening? Any other suggestions for responding to the nothing-to-hide argument?

My best counterargument for "nothing to hide": What do you think Nixon would have done with access to a limitless database of (effectively) every human on earth?

Privacy is important. But vastly more important is unaccountable power.

Playing devil's advocate here.

But that has never happened and probably never will. Nixon was caught quickly. Besides, that wouldn't even affect me, only people who do bad things or are in positions of political power.

It seems to be oft-overlooked that Manning and Snowden have managed enourmous breaches of what is supposed to be secured information (yours and mine, and probably everyone elses!). Staggering breaches. Any they're boh essentially nobodies - Snowden was an army washout who became a securtiy guard who has penetrated the US security apparatus in a way that would have been considered the greatest KGB coup ever if it had been a cold war operation.

There's a staggering lack of basic competance around protecting this stuff. The CIA director who lost his job over Aimes must be wondering what the modern mob have to do to get fired.

Interesting; it appears someone failed to redact some data from the slides. In the Facebook chat example, the message is "to" 1536051595.

Using the Facebook Graph API, we can gather information based on this ID: http://graph.facebook.com/1536051595

Which leads us to the Facebook profile (https://www.facebook.com/arash.gorjipour.5) of an individual, real or contrived, named "Arash Gorjipour". His email address and phone number are all exposed in one of his uploaded photos: http://i.imgur.com/0UUk5cB.jpg

I wonder what the reason for this man being in these slides is.

Probably picked at random. The screenshot 'test case' could have been you. We are all in the database after all.

Of course, it would have been more amusing had the 'selector' been chosen as 'Barack Hussein Obama II'.

Forget amusing, that would have been a perfect action trigger: people are OK with privacy infringement on others, but when it happens to them, they are more likely to be upset.

I suspect, or maybe just hope, that politicians are protected in some way from this. While it is unfair, at least it would mean less opportunities to extort or threaten lawmakers. Though, obviously, it would be best if we ALL were safe from that kind of crud.

I think it's a test account. The text string which reads something like 'does it still recognize me?' is very much like the kind of thing I'd type in my QA days when I was testing a new system.

If I were putting together a deck on that system I'd also probably favor test data over live data, if for no other reason than it's easy to come by.

They may have used an existing public profile since he's already displayed it openly. He's a realestate broker after all so, presumably, he's got "nothing to hide".

Anna Chapman was a realestate broker too...

In keeping with that line of thought would it not be better to redact the information you are presenting? I don't see why you need to write it out in full.

You could say 153xxxxxxx and "Arxxx Goxxxxxxx" just to be sure and if you need to post links you could use a URL shortener.

Probably because he has dark skin, and his native tongue is probably something other than English.

I'm a little worried now because I visited his page, and this will surely be logged, hence my past online activities may now be investigated.

Don't worry, you were already on their list.

HN fields roughly 200,000 unique visitors each day, most of which have a markedly anti-gov't-spying slant[1], that's enough evidence to be in their cross-hairs.

[1]: Such that in some capacity you might participate in the creation/promotion of methods or software to get around their snooping technologies.

Yup. I think that we all classify as "Enemy sympathizers". I wonder when we will be classified as "Enemy combatants?". As soon as somebody mentions violence, I suppose.

Oh bum..

Thoughtcrime.

Quite.

Now lets all forward to newspeak in schools instead of English in the coming years

Did the Guardian change the slide? I couldn't see that slide, and it has now appeared with a value of 1234567890.

The relevant slide is inline in the article, under the first appearance of the string "facebook". It was apparently redacted by the gaurdian; see nwh's link to the archived page below.

do you see that in this document? i can't find it anywhere and wonder if the guardian subsequently redacted it:

http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

There's a small version of the original slide here, which has since been removed — http://archive.is/xcwg6

He's Canadian. And he has dark skin and a funny-sounding name.

He's (almost certainly) a real person, by the way. I called his office. He wasn't in, but they offered to page him for me.

Just FYI (almost certainly of no importance because this individual was chosen at random for the slides): his name (both first and surname) are Persian. I'd guess he was an Iranian (graduate) student who has decided to stay in Canada after his studies; possibly to be "free" from an oppressive government's espionage and meddling in his private life. The irony...

This is brilliant, I love the screenshots:

Foreignness factor:

The person has stated that he is located outside the U.S.

Human intelligence source indicates person is located outside the U.s.

The person is a user of storage media seized outside the U.s.

Foreign govt indicates that the person is located outside the U.s.

Phone number country code indicates the person is located outside the U.s.

Phone number is registered in a country other than the U.S.

SIGINT reporting confirms person is located outside the U.S.

Open source information indicates person is located outside the U.s.

Network, machine or tech info indicates person is located outside the U.s.

In direct contact w/ tgt overseas no info to show proposed tgt in U.S.

It's quite easy to lose the protections of a U.S. citizen indeed!

The person is a user of storage media seized outside the U.s.

Interesting, so everyone who ever hit a MegaUpload link is potentially a foreign entity?

Kind of puts into perspective why they would coordinate such a massive raid on Megaupload. The target may not have even been the data - merely seizing the data puts anybody who has accessed the megaupload website as an easy target.

You crossed the tinfoil line. Copyright infringement was sufficient motivation for the actions taken. The megaupload raid was not okay, but I am pretty sure Hollywood was behind it, not the NSA.

Just a few years ago this very article would cross the tinfoil line. Plus, don't be naive to think that the government wouldn't use an accusation of committing a crime to cover what they really want to do.

For instance, need data from a server's hard drive? Accuse someone you know who has data on that server, not necessarily the data you want, to have an excuse to seize said hard drive and analyze it. Nope, turns out the accusation was incorrect, here's the hard drive back. Ah, is getting other data not covered by the warrant illegal? It just might be, but you can't complain if you don't know they did it and you probably don't have standing to sue over it to find out. Plus with authorities able to get double-secret warrants based on triple-super-secret laws issued by not-so-secret courts with "you can't even admit you were here" secret proceedings, how would anyone know in the first place?

Remember, government agents have the authority to lie to you in an effort to complete their goals.

Not that I'm saying the NSA was behind MegaUpload or anything, just saying it's feasible.

You've got a lot of nerve impugning tinfoil these days.

Well, there are reasons to put on our tinfoil hats now... heck, last I heard, MIT students had managed to inject memories into mice.

The headline for that article was astoundingly misleading. They created a fear response in the mice to a place that the mice had never been.

Hollywood by itself had absolutely no chance of reaching across to new zealand and persuading the NZ police to break NZ laws to arrest him.

Just to be clear, Kim Dotcom was a NZ resident, and had broken no NZ laws.

At this point it would be a bold man who made the claim that the NSA had nothing to do with investigating a foreign person and/or their company, tracking that company's international internet usage, monitoring their involvement in possible illegal activities and providing that information to US authorities who could use it to reach out across the world and attempt to have that person extradited to the US.

In fact, I cannot understand for a second why you are trying to make that claim?

@netrus - I am not even sure where the tinfoil line is anymore.

At this point, the line itself is even made of tinfoil.

Aaagh! Everything is tinfoil! Where did this tinfoil even come from! Get this tinfoil away from me! Aaaagh! Heeeeeeeelp!

Having a 3 1/2" floppy sent to someone outside of the country 15 years ago will probably count as well.

> It's quite easy to lose the protections of a U.S. citizen indeed!

That, coupled with the fact that they only require 51% certainty in the foreignness factor makes me think this is intentionally designed to make every single person they come across a subject to surveillance.

I can see Weasel terms like "use of storage media seized outside of the U.s." be extended to mean pretty much anything.

The fact that they don't require a wiretap order or even a warrant to monitor foreign citizens is disturbing in itself and is based on a questionable internal interpretation of the law.

On cspan this morning, the phrased it as: a secret interpretation. the fact that those two words can even sit together hurts my head. How do you secretly interpret something one way, but openly interpret another way.

What is really interesting here, is that this disproves what has been said. EVERYBODY'S DATA IS COLLECTED, but to query the data of a US citizen you need to simply provide a 'mitigation reason' as to why you accessing that data.

That then provides an audit trail, where something, or more likely, nothing is done to check that decision was valid,.

My favorite part is that the form assumes you need only a single line for the "Justification" field.

Holy shit... Apparently, the only way to ensure privacy is to go Stallman. Funny how yesterday's "conspiracy crackpot" became today's visionary.

Stallman never was a conspiracy crackpot, he always was a visionary. The only thing that changed is some people's judgment of him.

Exactly. It has been obvious and widely understood for years to anyone who has ever used a network analyzer that systems like this could be built. The question was always would they be built. Stallman, and others, bet correctly based on their better reading of history and human nature.

You can be completely correct and still be a crackpot.

What we need is strict limitations on what can and should be collected, and how it's used, plus better methods of securing what's being exchanged. For example, sending email as plain-text, leaving it on the server as plain-text, maybe that's a bad idea.

The NSA isn't necessarily the only reason you'd do this. Foreign governments are going to take an interest in this, too, and it's only a matter of time before someone gets access the data the NSA is hoarding. No program of this scale is ever 100% secure.

> You can be completely correct and still be a crackpot.

This is very important. What do you mean by "crackpot"?

It's not feasible for the average person to restrict their lives to the point that RMS does and advocates for.

* Reading the web via email only

* Using completely free software and hardware (which as far as I can tell, limits you to a very small subset of Linux on a single Chinese-made netbook)

* Not carrying a cellphone

* Not using any social networks.

Stallman's principled stand is admirable, but untenable for most. I need to violate every single one of these tenets in an average day at work.

And that's before we even enter the realm of entertainment, which is even worse as far as the FSF's definition of freedom goes.

Principled != crackpot. Crackpot is an insult intended for the feeble minded and is used to reduce any opinions a person might hold on a subject as reject-able out of hand.

Over unity energy generation from the vacuum is rightly labeled as 'crackpot' imo, Stallman's position, while extreme should (again, imo) not be labeled as such.

Where Stallman breaks from admirable principle and dives into untenable crackpottery, IMO, is where he calls proprietary software evil.

Crackpot => unsupported by evidence.

Calling proprietary software evil is an opinion, and there are plenty of examples of evidence that proprietary software was created in ways that one could label as evil. Give it a while and there might be some revelation which will cause lots of people to go 'oh, that Stallman was such a visionary, calling proprietary software evil'.

Now on this particular aspect of Stallman's reasoning I find him hard to follow because that would mean a whole class of something is bad whereas I believe it should only apply to instances on a case-by-case basis. But I'm going to hedge my bets here and sit it out for the next decade or two (assuming I have that much time remaining) to see if he might not be on to something again that is still hard to see from where we are standing right now.

One way in which this could play out is that in order to avoid certain societal fates is to have nothing but open source for certain classes of application (for instance, voting computers, software in use by the government in general or software that is used to power network infrastructure).

Don't be too quick to judge, Stallman has been right more often than I'm comfortable with on some of his most 'extreme' views.

I've never heard Stallman be right about anything that wasn't blindingly obvious to anyone who was an open-minded observer of the same things at the same time.

He's not the only one that's been crowing about electronic surveillance. Ever since things like Carnivore (http://en.wikipedia.org/wiki/Carnivore_(software)) were uncovered in the 1990s, it's been obvious that there's a lot going on we will never be fully informed about, that the internet is no longer a safe playground devoid of malevolent actors. Mailing lists and USENET groups at the same period of time were constantly aflame with these sorts of issues.

If you can cite an occasion where Stallman has had a unique insight into the situation, I'd be surprised.

Stallman, for all his posturing and relentless drum beating, which is at least admirable from the point of dedication, is still no Alan Kay, Marvin Minsky, Marshall McLuhan or Raymond Kurzweil.

Moral judgements are subjective opinion by nature, fair enough, but I bring the crackpot label in for exactly what you say, thinking in absolutes, in black and white, instead of nuance.

In the real world, that shows a distressing lack of critical thinking and a further distressing abundance of dogmatism.

"Proprietary software is bad" -- Subjective value judgement.

"Properitary software is evil" -- Subjective value judgement that shows a lack of thought.

"You should always use free software wherever possible." -- Subjective value judgement.

"You should use absolutely nothing but free software ever" -- Subjective value judgement that shows a lack of thought.

I mean, the FSF "disapproves" of software that is completely free on its own (Fedora, Firefox), merely because they point out nonfree things you can use. (Fedora's firmware bundles and some repos, and Firefox's addons site).

That's completely idiotic. Apparently the FSF's "freedoms" do not include the freedom to run whatever software you choose if it's "unfree".

The proprietary software as evil thing comes as a morality judgment, that the potential evils from such software/licensing far outway whatever positive nuance it could bring to the table. A nuanced reading of the past 75 years of copyright/patent law and judgments can come to the conclusion that such an ecosystem is detrimental to the rights and ability of end-users and developers.

Guess what the solution to the proprietary software problem is? Not using or promoting proprietary software or platforms that enable it.

You are getting upset that the Free Software Foundation has standards to be met to consider software as "free". To dismiss their agenda as existing in 'crackpot' territory is invalidating a legitimate argument to support your shaky conclusion.

> Over unity energy generation from the vacuum is rightly labeled as 'crackpot' imo

Then it seems that crackpottery is a term that may be removed in retrospect. I'm sure at some point in the future someone will crack the energy from the vacuum riddle, who knows.

* RMS reads the web via email because he's traveling virtually all the time and rarely has Internet access. A batch-based system makes more sense for him. This isn't an ethical stance, and the fact you include it hits your credibility severely.

* The FSF uses computers other than Yeeloongs. The FSF also doesn't really care about free hardware. The Yeeloong has chips with non-free firmware burnt in, and the FSF doesn't care because that isn't software. It's the Free SOFTWARE Foundation, after all.

* Stallman is on a few social networks, notably identica @rms@identi.ca (possibly now defunct). He probably has a GNU Social endpoint.

I think you're conflating Stallman's willingness to be uncompromising in his own lifestyle with his calls for reform. Stallman is fairly intelligent and understands that not everyone can live like he does, but I suppose he feels the need to answer the question of "what should you do in the present beyond push for reform."

I also don't know what "entertainment" you're talking about. The FSF is against proprietary video game engines, but their mission pertains to software, not music/movies/etc.. They campaign against DRM because DRM requires non-free software to enforce.

RMS reads the web via email because he's traveling virtually all the time and rarely has Internet access

Surely you can't expect people to take this argument seriously. It's easy to get internet access on the go in much of the world already.

RMS emails in restaurants, cars, trains, etc., in Europe and the United States but also frequently in SE Asia and South America. There are pictures of him responding to email in the mountains in Nepal.

It's easy to get Internet access on the go in most of the places I've been to, but I've been to a tiny fraction of the places RMS has been to.

I didn't say one should never use offline mail. I'm just disputing that it's a sensible default, rather than a backup.

An example of a crackpot is Glenn Beck, that is, someone who is drawing incorrect, incoherent conclusions from the facts they observe.

Suggesting that people abandon social networks, never own cellular phones, avoid using the web almost entirely, these are extreme positions. What makes them crazy is when he's an advocate that everyone should follow these edicts.

Surely it's some kind of "geek social fallacy" that's being applied here. Stallman has come up with what he perceives as the optimal strategy and anyone who diverges from this is doing it incorrectly, just as how free, open-source software is the only kind of software that's acceptable, and everything else is "evil".

I think the free access to the data once it's mined is worse than the collection. Such access should require a warrant, if not a wiretap order, not a justification one-liner.

"You can be completely correct and still be a crackpot."

Not really. At that point, you are just using the term as an ad hominem in a childish attempt to ward off cognitive dissonance.

You don't win an argument by calling the other guy a weirdo.

His observations are correct, but his conclusions are incorrect, just as people like Glenn Beck start out with facts and end up with paranoid delusions and fantasies.

I think Stallman's observations are valid, but his method of dealing with the implications of those observations are impractical, if not completely wrong.

When you are done attacking Stallman with a false analogy, would you care to name a few of his invalid conclusions?

More specifically, what is so impractical or "completely wrong" about not using smartphones?

He's not opposed to smartphones, he's opposed to cellular phones as these can serve as a tracking beacon, following your movements.

Given that the cellular providers are capturing and archiving location data, this is fact, his conclusion is we should avoid using these sorts of phones completely. Why? The reasoning here is a awfully thin, but has something to do with "being tracked = bad" and then goes into crazy territory from there. It's the same thing with credit and debit cards. They can be tracked, therefore bad, therefore nobody should use them.

If he's concerned about remaining invisible, then this must be applied rigorously across all aspects of his life. Does he wear dazzle face-paint or glasses with bright IR LEDs on them so that CCTV cameras can't pick him up? Does he only use methods of travel that require no identification? If the FBI wanted to retrace Stallman's activity on any given day, it'd take hours at most to piece it together.

The sign that someone's a crackpot is in how inconsistent they are in applying what they've concluded. It means they're missing something important.

For example, there are people that have a genuine need for absolute secrecy, that need to remain invisible, yet they still use cellular phones, email, and social networks. They're aware of the same risks as Stallman, but they take precautions instead of avoiding them completely.

It's notable that Osama Bin Ladin was taken down because he'd gone to such great lengths to avoid being tracked that he stood out as an anomaly, an approach that proved to be self-defeating. He had this large house, but a paranoia about electronic snooping so severe that he had no internet connection, and that alone made that house highly suspicious. If you're that affluent, you have an internet connection, even if you barely use it.

Everything Stallman advocates to avoid detection just makes him an even bigger target.

> Why? The reasoning here is a awfully thin, but has something to do with "being tracked = bad" and then goes into crazy territory from there.

You don't understand why tracking may be bad? Or are you just trying very hard to mock his very valid conclusion?

Here's other people's thoughts about cellphone tracking: http://www.zeit.de/datenschutz/malte-spitz-data-retention/ (totally crazy, right!)

> If he's concerned about remaining invisible, then this must be applied rigorously across all aspects of his life

No, it mustn't. Every bit helps.

> Does he wear dazzle face-paint or glasses with bright IR LEDs on them so that CCTV cameras can't pick him up?

Perhaps he does not yet live in an area with seamless CCTV tracking.

> The sign that someone's a crackpot is in how inconsistent they are in applying what they've concluded. It means they're missing something important.

You must be a crackpot then because you're clearly missing that Stallman has probably managed to avoid having his daily movements tracked by some carrier.

> Everything Stallman advocates to avoid detection just makes him an even bigger target.

To whom, with what (crackpot-like) line of thought? Stallman is very open about his principles, his reasons and his actions. It would be extremely dumb for anyone to derive from this information that he is dangerous or a worthwhile target.

Tracking can be bad for some people, it can ruin their careers, destroy their marriage, completely upend their life if that sort of information got out. However, for most of us, it's not especially valuable information and any one day will look like any other.

When I engage with social networks, use a cellular phone, I'm aware of the liability. I'm making a conscious trade-off. I really would like it to be less of a big deal, that the privacy implications were minimal, but this is the world we live in. I support political parties and representatives that would restrict how this sort of information can be used, making it less likely to be collected in the first place.

> No, it mustn't. Every bit helps.

Either you're trying to avoid being detected, or you're not. There's no half measures here.

> it can ruin their careers, destroy their marriage, completely upend their life if that sort of information got out.

> I'm making a conscious trade-off.

No, you're not. If you and the people who have had what you wrote happen to them (they obviously would have been more careful than you) were making conscious trade-offs, nothing bad would have happened to anyone as a result. In fact, you do not even know what information you are disclosing to FB (it's more than you are writing) and other, unknown to you, parties, so a conscious trade-off is impossible. You are just patting yourself on the back for being satisified with your ignorance.

> Either you're trying to avoid being detected, or you're not. There's no half measures here.

From what I understand, he is refusing to provide personal information to a carrier and possibly other unknown parties, because that is potentially harmful and not beneficial in any way to him. Why are you insinuating that he is trying to avoid detection, as if he were some criminal? And by the way, even criminals aren't stupid enough to do everything wrong because they cannot do everything right.

I don't use Facebook specifically because of their habit of leaking information to anyone and everyone. I do use other "social networks" where I'm not obligated to provide a dossier on my life.

I've even got Facebook's site and associated flam blocked on my computer so I'm not bombarded with their inane commenting system, "Like" buttons, tracking features, or other garbage I want nothing to do with.

I'm taking a risk by using a cellular phone, I understand thins, however I believe the down-side of using one is better than the down-side of not using one. That I'm not a politician or celebrity factors in to this decision.

I'm not even sure what Stallman's full reasoning is behind cellular phones as it's always glossed over with some kind of hand-waving about tracking.

"Tracking can be bad for some people, it can ruin their careers, destroy their marriage, completely upend their life if that sort of information got out. However, for most of us, it's not especially valuable information and any one day will look like any other."

> I think the thing to realize here is life can change very quickly. What if, for one reason or another, you become a celebrity all of a sudden - Or happen to acquire particularly well-connected enemies. When this kind of powerful info is used against you things look quite different.

Stallman's stance is against all cell phones, not just smartphones. And I'd argue that in 2013, to the point where we're issuing basic phones to welfare recipients for the purpose of job searching, that this is an invalid conclusion.

As is only using the FSF's definition of free software (where it matters less that the software itself is free, but that the software doesn't point out to you any nonfree addons. Fedora Linux is free software, as is Firefox but since they allow nonfree firmware blobs, and addons respectively, they don't count).

Or free hardware, Good Luck With That, unless you like a single netbook made by a single company in China.

There's been many missed opportunities to get truly open hardware, an to this day we're still missing out on them. There are initiatives to remedy this, but they're still far from complete and need more motivated drivers to carry them forward.

Using a crappy computer from some no-name company in China is a protest vote and is not pushing things forward.

On the other hand, getting hardware hackers together to create a 100% free hardware platform would. The Raspberry Pi is close, all that's really needed is for some more aggressive lobbying to get the PowerVR driver component open-sourced.

Or consider, given how people are taping out custom Bitcoin ASICs, why is it inconceivable that someone could tape out an open-source CPU?

I fail to see how not owning a cellphone, only using free software and suitable hardware puts me at a greater inconvenience than, say, having all my life (movements, communication, interests) digitally recorded and made available for later arbitrary use (by any type of government we might have ...). I honestly wish I had the willpower and independence to pull it off.

On the other hand, I totally understand the people who firmly believe that neither governments nor rogue personnel will ever abuse this information to their disadvantage. After all, billions of people firmly believe in some arbitrary deity and we haven't managed to prove them wrong.

That's a laughably false dichotomy. Using free software does not in any way guarantee that you won't be tracked online.

>As is only using the FSF's definition of free software (where it matters less that the software itself is free, but that the software doesn't point out to you any nonfree addons.

You're conflating the FSF's definition of free software, and the FSF's criteria for recommending software to users.

The FSF sees Firefox as free software (now that the proprietary error-reporting system they used is removed); they won't recommend Firefox, because it recommends non-free software. Fedora is a distribution, not a specific program, and they won't recommend it because it recommends non-free software.

By the FSF definition, a license is free if it protects the Four Freedoms; but software licensed under that could be something the FSF doesn't wish to endorse.

You make crackpot sound more like a personality trait than a status.

How does browsing the web via e-mails and cron jobs make for more privacy?

No javascript tracking. Very strange behaviour (therefore, less behaviour tracking). That's what I can think of.

Honestly, if the NSA wanted to know what Stallman was up to, they'd apply the $5 wrench technique (http://xkcd.com/538/). All the tin-foil in the world can't prevent them from getting what they want if you're suddenly a Person of Interest.

Is there no difference between specifically targeting a suspect and gaining physical access to their hardware vs. any number of government employees/contractors sitting at their desk browsing through anybody's data with little to no technical limits and little to no oversight?

One of the slides literally says that users must be careful to and their query with another parameter to avoid running afoul of the law.

At this point the only difference is cost and scale. What the NSA is doing needs to be reeled in big-time, probably even shut down completely, but that doesn't mean being all tin-foil hat will somehow make you immune to what they're doing.

I'm sure they know everything they need to know about Stallman, just as they do about everyone else, apparently. Unless he's sitting in a cave writing EMACS source on goat hides, they'll have a window into his activities.

> At this point the only difference is cost and scale.

Only if we are talking about the same types of attack, which we aren't. If you do "wrench" style targeted attacks at a large scale, you'll leave 10%+ of the population injured, how is that supposed to work out for a government?

Stallman's counter-measures probably work as long as only very few people use them. The same is probably true for terrorists, which is why this whole dragnet surveillance does not really work towards the stated goals and "crackpots" like me suspect it may have more to do with bullying people into self-censorship.

You're completely missing the point -- it's unfeasible, unpractical, and unproductive hitting millions of people on the head with $5 wrenches. This is the entire point -- they can do it easily with everyone now, they're not hitting people with wrenches -- that would invoke suspicion and retaliatory response that would curtail their legal powers to snoop around.

Maybe you're forgetting about the sorts of things that went on, are probably still going on, in various brutal military dictatorships around the world. Wrenches are just the start of what they do to people before they disappear them.

It isn't impossible to beat information out of millions of people. It's been done before and it'll be done again.

You say it'd invoke suspicion, but it wouldn't. If you're at the wrench phase of interrogation, you're already in a world where legal powers don't matter.

>Maybe you're forgetting about the sorts of things that went on, are probably still going on, in various brutal military dictatorships around the world

Are including the USA in the list of 'brutal military dictatorships'? Because the USA disappears people: https://en.wikipedia.org/wiki/Khaled_el-Masri

https://en.wikipedia.org/wiki/Rubber-hose_cryptanalysis

NSA spying is not designed for the individual. It's designed for the masses. It's to keep the populace in check. It's Century of the Self and Edward Bernays, except for the 21st century.

It's an update to what was already going on.

Sufficiently strange behaviour actually enables tracking; 'strange' is almost a synonym for 'unique' in this context.

He only runs code that he can vet. You can't do that when you use Javascript, or use a proprietary OS, for example.

This is by far the best guide I've found to do exactly that:

http://crunchbang.org/forums/viewtopic.php?id=24722

I haven't read all of it, but I'd recommend using LUKS instead of TrueCrypt.

Reading these slides, I'm trying to parse what these slides do or do not say. I'd like to leave aside the speculation about what the NSA is probably doing.

First of all, XKeyscore seems to be primarily about the frontend query interface rather than the backend data storage, at least as far as I can tell. It looks like you can basically query their database by email address and get a set of records (email, chat, http logs) back. It looks like there are separate tools for viewing specific records as well. I assume they're joining records on some combination of email address, IP address, timestamp, etc -- not unlike a modern ad server.

A few practical thoughts: * It's worth noting what's not shown in these slides. Specifically, I don't see any ability to query the full text of emails. The more I see about this, the more I'm convinced the NSA is not collecting email body texts directly from corporate servers. Facebook messages I'm less sure of. * How are they collecting HTTP data? I assume intercepting at network hubs? * Given that it appears that individual records are HTTP requests, I'm shocked at how few requests are in the database. 41 billion seems an order of magnitude smaller than I'd expect. Could it be a record is something else? * Interesting to note the "Miranda number" and "Foreign Factor" fields that look like ways of saying "yes, I have permission to do this." Might explain why a sysadmin could bypass these things but your everyday NSA analyst could not.

Note regarding the amount of items, that the presentation is from 2008, and they claim to only be able to store 3 days worth of full data capture.

Regarding ability to query the full text of emails, this program does not seem to indicate that it would collect the data directly from the services servers in anyway. But consider that they do indicate the ability to monitor web traffic at the protocol level. Capturing e-mail is no harder, so it'd be surprising if they're not.

It doesn't show reading full emails in the screenshots, but the sentence right underneath reads: "The analyst then selects which of those returned emails they want to read by opening them in NSA reading software."

One of the slides [1] has the full message text for a Facebook message. If they have it for Facebook, I'd be surprised if they also don't have it for email.

[1] https://image.guim.co.uk/sys-images/Guardian/Pix/audio/video...

Lurkers first post here.

I always said saying "I told you so." when stuff like this started getting revealed would feel like a hollow phrase. Some of us have spent quite a bit of time talking about these issues, and were mostly rejected as crackpot "conspiracy theorists". While there are plenty of those around, maybe I could use this slight moment of pseudo-clarity to propose something.

I could tell you where this is going (removal of ex post facto, and eventually algorithmic based pre-crime), and who is largely behind it, but once again most of you would probably perform the standard knee-jerk reaction against "conspiracy theory", only to wait around and repeat the same kind of stuff you are saying now, whenever the next steps are put into action.

We curious geeks have been too cocky, always thinking we could use our superior knowledge of technology to beat "the man". Well boys, the man is learning our tricks, and he's starting to get better at them than us...

The NSA is but a cog in a greater machine, and until we all realize that and start conversing on what/who that machine is, we will continue to spin our wheels uselessly.

Happened also because some of the geeks rented themselves out to the NSA.

Just the MICC in general. I remember talking to Mudge on the phone about two years ago, asking him how Darpa was treating him. He said he loved it... guess where he is now? Google.

The problem with conspiracy theorists is that they believe their theories to be 99% certain instead of the most probable 1%. "I told you so" goes both ways except that conspiracy theorists never accept and admit it, there will always be another conspiracy to cover the current one.

In the overall game it simply makes much more mathematical sense to treat conspiracy theorists as crackpots since they will so often get it wrong. Even a broken clock is correct twice a day.

Negative. The problem with conspiracy theorists, or any group for that matter, is that they are so easily put into that group and then dismissed as a whole. Personally I think at the core of the matter is an enormous imbalance of importance placed on the authority or source of an argument, to the detriment of argumentative logos. Possibly some form of the genetic fallacy.

For example, you deride an entire "group" of people for believing their postulations to be 99% probable and assert that instead they are more likely 1% probable. I would say you just fucking pulled those statistics out of your ass, the very act for which you are deriding the group for! Such irony in a single sentence has been seldom found.

Sure, I agree with you that there are a plethora of quite frankly ridiculous conspiracy theories (not to be confused with the theorists themselves, who can often be just as ridiculous). That being said though, you are in fact completely wrong when you say "In the overall game it simply makes much more mathematical sense to treat conspiracy theorists as crackpots since they will so often get it wrong." and here is why.

Conspiracies are a driving factor throughout history. To ignore them in the past and in the present is to dismiss the very core their historicity. To dismiss them based on some flawed, arbitrary assertions about mathematical calculations that have no academic backing is simply daft.

Also, as I said before, there are indeed plenty of logically absurd theories... but why exactly are they absurd? One, they defy logic, but more importantly, the more absurd ones are often based on no evidence whatsoever. If you have such a difficult time sifting through the completely absurd to find the logically probable/possible in order to more finely tune your engagement, that is a deficiency on your part, and no one else.

I'm trying to think of an analogy to this... It's very much like the general reaction to homeless people. Many people have a general reaction to homeless people in which they assume they are either scammers, drug addicts, or lazy. They may have had a bad experience with a homeless person, and now seem to think almost all homeless people are not worth the time/effort. I understand how they can come to that conclusion, but the statistical facts do not support their argument, and, similarly, the statistical facts do you not support yours either, because you have essentially taken the same approach to "conspiracy theorists".

You are a conspiracy theorist. By definition, your powers of cognitive thinking are essentially retarded. This is what defines the conspiracy theorist.

Coincidence theorists, the ones that believe whatever they're told by whatever authority figure they believe in this week, are the ones with stunted cognitive abilities.

"Terrorists are the problem we must address" is no different from "Satan is trying steal your soul" or "Commies are gonna destroy our way of life" as a means of manipulating the more gullible people in a population. It's worked for centuries.

"You can fool some of the people all of the time, and those are the ones you need to concentrate on."

-George Bush, repeating advice given to him by Robert Strauss

OK, I will take the bait. What do you think the greater machine is?

Here is my take:

You do not need to posit an organized Illuminati-like conspiracy to have cause for concern. We can find plenty to worry about even if we limit ourselves to properties of the system that are either emergent or driven by natural human behavioural traits.

For example, a lot of people in positions of authority got there because they have authoritarian instincts, and seek self-validation not only by dominating and controlling others, but by ensuring that their position of authority and dominance is recognized by others.

This is very human, and very instinctive, and operates at an unconscious, almost sexual level. The alpha male will seek to dominate the pack and to remind competing males of his superior status. You do not have to consciously be aware that you are seeking power, money and sex, but you are, nonetheless.

This instinct can operate both consciously and unconsciously. Those who make decisions to concentrate power and authority, to separate and elevate themselves from the general population - they do not have to be consciously aware of what they are doing. They can and will rationalize their beliefs and actions to make it fit in with the dominant culture of their peers. This process is called confabulation (http://en.wikipedia.org/wiki/Confabulation) and everybody does it all the time - it is the only way that we can make sense of our lives and live in a human body without going insane with the sheer irrationality of it all.

These instincts manifest themselves in lots of small, individually inconsequential decisions. Normally, this is OK, because our social and bureaucratic technologies are (were) too ineffective for too much harm to be done. The ongoing march of modern information technology, however, looks likely to change that, meaning that the unconsciously malicious instincts of humans in positions of authority can become amplified and magnified.

I would be particularly worried if this resulted in a feedback loop - so that increased power and increased power-seeking behaviour mutually reinforce one another in a runaway process. I cannot readily identify such a loop in operation though -- can anybody else?

I hope I am not responding to too many people and seeming spammy, but here goes.

I agree completely that we do not necessarily need to posit and organized "Illuminati-like conspiracy" to have cause for concern. There are plenty of studies showing increasing likely-hood of sociopaths rising to the top of power structures, and is often just due to how to system as an autonomous entity functions.

What I do posit though, is that, in fact, there is, borrowing your own term for lack of a better one, an "illuminati-like conspiracy". I have been considering an attempt at scholarly paper on the matter for some time now, but let me try to be terse and possibly just point you in the right direction, because I don't think I'm quite prepared to defend the full assertion in public yet.

I will start with your question about power feedback loops. Here is a paper regarding the global network of corporate control that anyone interested in the global power structure should read. http://arxiv.org/pdf/1107.5728v2.pdf

I even contacted one of the researchers (Glattfelder) during the Libor scandal, wondering if we could use some of the new information to analyze the scandal better. He said it would be extremely difficult due to how good the companies are at obfuscating their dealings.

Now, as far as the conspiracy, I would like to point out one thing. I do not claim that there is but a single conspiracy (a trap assertion many fall into making), and instead would say there there are but a small number of very powerful ones operating at any one time, sometimes in competition and sometimes cooperatively. Regarding the "illuminati-like conspiracy" itself, I have one primary reading source for you, if you are genuinely interested in the subject. It should be enough to get you started on the more serious analysis of what I am talking about. http://www.amazon.com/Anglo-American-Establishment-Quigley-C...

This seems relevant: http://www.zerohedge.com/news/2013-04-26/illuminati-were-ama...

One reason I love HN is due to the contributors having better than average critical thinking skills (IMO). Unfortunately that isn't enough when it comes to asking someone to believe that all governments might be controlled by a secret organization, or something equally as sinister and unbelievable.

Since we know that our (USA) entire financial system is backed by (Federal Reserve) bankers that loan money without moral guidelines, it could be conceived that a family like the Rothschild are at the top of the world power hierarchy. What do they always say in the detective movies, "follow the money".

> The NSA is but a cog in a greater machine

Indeed. I always follow The Money because that's who owns the government.

America, including the NSA, is owned by the same small cartel that have the monopoly on the issuing of our currency and credit. Coincidentally, they're the same cartel we're "indebted" to.

Most people have never heard of the four largest banks in the world:

http://www.northerntrust.com/documents/white-papers/asset-ma...

At this point the only thing protecting the US (and the world) from the worst tyranny imaginable is that USG's essentially unlimited power is wielded by individuals (rank and file workers, career bureaucrats, political appointees, and politicians) with a fairly reasonable sense of morality and belief that they are constrained by both the constitution and morality.

I'm don't think that is a stable long term system. Either some effective limitations (technical and political) are put in in the next several years, or a few decades of "us vs them" and self justifying security crises will produce a horrible result.

Absolutely.

Now, the thing that prevents that from happening is money.

So much money is concentrated on so few people that it protects itself and the owner becomes invulnerable. Add to that that too much money very often corrupts its owner. The predictable result for society seems pretty obvious.

You're spot on about money. I saw Keith Alexander's talk at the recent BlackHat conference, and was miffed at people applauding the guy, but then realized that it's a Black Hat conference, the epicenter of the monetization of exploits.

The other factor is that the NSA's reputation is irreparably tarnished, and they will continue to attract the wrong kind of people. I think we need to prepare for some dark times ahead.

Honest, maybe naive question, but what types of programmers actively help build and maintain systems like this? I turned down a job for a company that is less than a mile from my house because I viewed their business as immoral.

Hard for me to fathom anyone taking a job, helping to build systems like this. I get that many of the components of a system like this could be seen as harmless. However, a system of this complexity must have some talented engineers bringing it all together and making it work. How can they feel good about what they are doing?

I guess some people are in it for the money. Some probably like the technical challenge. But I'm sure there are also quite a few that believe this system really is preventing terrorism, following the law, and not doing anything wrong.

There's something more important than money and technical challenge.

It's called Power.

I can now look into any person's email and whatever legally. You and I may get a squishy feeling in the stomach about this but I can tell you that 1 out of 2 people will be OK with this if they aren't called out. (like watching porn, most people won't admit in front of kids and their parents but they will do it because they think they won't get caught)

They're probably not thinking of it as immoral. Most likely they actually do believe they're improving national security by doing this.

Or they just like the paycheck, that's a big possibility. I imagine the NSA is probably happy to pay a lot with a large amount of bonuses to keep people in roles.

This sentiment was also echoed by Snowden in the original video interview. He supposedly had greater insight as to how civil rights at large were being violated, only because as a sysdmin he (and not others) had a broader picture of what was going on, and at what scale it was going on.

So, it's easy to imagine NSA recruiters coaxing potential hires with convincing lines like "we have internal courts, procedures, and checks and balances systems to prevent abuse of citizens' privacy".

In a profile on Palantir (which sells tools that help with this sort of thing), one employee was quoted as saying "this really is about saving the Shire". Had to laugh.

I remember that (I think it was in Wired), and it struck me as a basic naivety on their part as when they were building it they were thinking about catching bad guys without realising the line between who's a good guy and who's a bad guy is determined way above your head.

In the spirit of the 'Shire' quote they should have also realised 'with great power comes great responsibility'. No one has demonstrated that they are responsible enough to have that level of power over millions of people.

The NSA sells the job to naive college students on patriotism and generic descriptions of working with cutting edge cool stuff. Once you're in, it's easy to rationalize that all this will be used only for good. Especially if the alternative is quitting your stable job that supports your family.

Many of the low-level analysts are very smart enlisted personnel with ranks of (E1 to E5) from among the various services. Many of them use the GI Bill to pay for college. Some stay in the IC, while others move one to different careers.

> How can they feel good about what they are doing?

You'd be amazed.

https://twitter.com/_nothingtohide

Seems to me that people aren't too concerned of nameless, faceless people looking through their stuff so long as people they know (parents, spouse, friends..) don't see it.

You know, probably one of the best things that could happen is for someone to pour out nsa data indiscriminately and en masse in order to just outright humiliate hundreds of millions of people.

THAT certainly wouldn't risk an apathetic response.

Nameless, faceless people can show it to their parents, spouse, or friends, or threaten to.

Unfortunately most people don't make that connection and assume that it "would never happen to them" and don't care if it happens to someone else.

It would be good if people realized how snooping is connected to poverty. In my scenario, snooping is used to gain business secrets. These are used to undercut businesses, resulting in lower wages and poverty.

E.g. Chinese hackers steal plans for American-designed products. Instead of Americans working, building and selling the things we consume, inexpensive imported versions are available b/c those companies didn't pay for the overhead of design and didn't take any risk. We need to protect ourselves from this.

One of my first jobs was at a company that sold and serviced surveillance equipment to school districts. It's a slippery slope where you can first focus on the good aspects -- murder cases solved -- but eventually encounter information that you have to make an ethical judgement on wether you are willing to play a role in implementing such technology. For instance, testing began on systems that could capture both audio and video and I became uncomfortable because kids say stupid things that can be taken out of context. I make no claims that audio surveillance was installed, but the casual nature of doing R&D on such systems without considering the implications became too uncomfortable for me to take part of.

The opportunity to work on a cutting-edge technical problem at scale most would never get to touch. That's a very tempting proposal. The secrecy itself probably also appeals to some folks.

I'd also imagine that a lot of folks only work on one small part of this, that in and of itself, is not objectionable. Where it becomes scary is when you expand the scope to cover every US citizen regardless of wrongdoing.

Times are hard and if it's a choice between your personal morals and feeding your kids or ensuring that your home doesn't get repossessed then there are likely to be plenty of programmers willing to make the compromise.

I have "friends" (read: we mostly argue about this) who fit your description.

They will tell you that there are rules in place to prevent spying on Americans (and if you take a look at the Foreignness Factor screenshots, there is a sense in which this is true)

They will also tell you that the benefits outweigh the cost. Here we have a system that has allegedly caught 300 terrorists, and they would tell you that spying on foreign people to catch 300 terrorists is a good trade.

I disagree with them on both counts, but you asked what they would say.

if you take as given that the NSA has prevented 300 9/11 attacks from occurring, you may feel it is justified.

surely you don't believe the premise, but as a thought experiment pretend it were true. would the NSA actions be justified?

Okay, so if ~3,000 people died in the 9/11 attacks, and we prevented 300 of those, then that saves 900,000 people.

But in order to save 900,000 people, it required a bold attempt to eavesdrop on all ~1.3 billion people currently using the entire internet, leaving us with an efficiency of 0.1%.

Not only that, but the process needed to occur continuously, for a decade, ramping up, over time.

But wait! It took nearly 20 hijackers to accomplish one 9/11, so that means it only stopped 15 9/11's in a decade, or rather 1 and a half per year.

So that means we saved 4,500 people a year, by eavesdropping on over one billion each year.

But remember, when you tally up all those ten long years of life saving:

Those 45,000 people weren't just ordinary human beings...

They were Americans. They were Freedom.

And at the end of the day... well... ✼sniff!✼ I think you and I both know that you can't put a price... on FREEDOM.

I have asked this same question myself. It's stunning that only one sys admin has come forth with this information. Further, politicians and higher business officials in companies usually don't know what computers are capable of, which leads me to believe that it's actually programmers who are suggesting that such horrifying systems be built (for example, no politician is going to say "let's make a copy of every piece of internet traffic everywhere." The programmer says "We can copy everything," and the politician says "Ok, do it"). The only excuse I could imagine that a programmer would tell themselves is "This is just my job, and I'm just building the software, but it's the government that's going to be using it, not me, so I've done nothing wrong."

You could say the same for many jobs: being in the military ("i am just following orders") making bombs making atom bomb human cloning

bottom line is anything can be justified if you want to justify it.

> How can they feel good about what they are doing?

Didn't you hear? The system is used to help track down bad guys who want to hurt America.

It sounds trite, but people really and truly believe that the issue is just that simple.

The internet is a public space. Unless you're using full client-side cryptography, one should not assume their activities are private, just as one doesn't assume their physical activities are private if they're done in the open in the public square. Using client-side crypto is the equivalent of entering a private residence; you can't just sit out in the open and expect privacy, you have to take special effort to keep your activities private.

You might want to have a look at the HOPE9 keynote with one of the first NSA leakers:

http://www.youtube.com/watch?v=FOFtQ6n3WR4

Clearly a very smart guy, that went very far in the NSA -- and for a long time felt he was doing "the right thing" -- but eventually quit because of what the NSA were doing.

edit: He also touches on how compartmentalization leads to people not knowing what they're actually working on/how it will be used in some cases.

Is this information the NSA could've attained left to their own devices, or is this sort of stuff only accessible with help from service providers, eg Facebook and Google?

What I'm trying to ask is: with all the hullabaloo Google, Facebook, Yahoo, Microsoft, etc have made about individual, manual reviews of information requests, are we still being lied to? I suspect that we obviously haven't been told the whole story by these companies, and that they are a lot more implicit in this than they let on, but this article seems almost like definitive proof that they did indeed allow unlimited access to user information.

If this is saying what I think it's saying, then I feel seriously back stabbed by the startup darlings -- Zuckerberg, Brin and Page, etc -- that so many people here love and idolize. They should absolutely be held accountable.

If you can tap large Internet connections then you can siphon off all HTTP and SMTP traffic and from there it's trivial to reconstruct sessions and from there get application level stuff.

I was doing similar things in the mid-1990s on shared Ethernet. It's really only a question of speed and scale and then of writing code that recognizes particular traffic (such as "this HTTP connection is a Facebook chat session").

Interesting. So then this can be done with zero assistance from service providers? Could providers have taken any steps to render that stream of information inaccessible? And if so, is it a costly effort?

Yes, use HTTPS for everything. It's not a surprise that all the logos in this PowerPoint have since moved large portions of their traffic to SSL. SSL isn't perfect (you can still see what domain someone is requesting), but it does prevent a lot of the snooping outlined in the presentation (without vendor participation, it's always possible that Facebook is siphoning off their messages).

Why would we assume that TLS is safe? The NSA could just as easily compromise the CAs and get all the certificates they need.

Not exactly. Compromising a CA would let them fool a browser into thinking that a fake Google certificate is a real one. However, if Google were diligent, they could publish their valid cert signatures anywhere they like, and users could check the signatures of the certs that are presented as genuine.

The TSA can't crack or impersonate a cert at will; they can only 1) try to trick you into accepting a phony one or 2) demand/steal the private key from the site.

Wouldn't having the private cert allow you to decrypt all communications encrypted using that cert?

Traditionally you generate an SSL public and private key, and send only the public key to the certificate authority for signing, so compromising the certificate authority doesn't give you the private key.

It does however give you the ability to issue yourself new public keys to conduct man-in-the-middle attacks [1]. If you compromise the same CA as the site whose traffic you're trying to intercept, you can bypass certificate pinning which is supposed to detect MITM attacks. So for example you can MITM gmail without certificate pinning detecting it if you compromise Verisign, Equifax or GeoTrust [2]

[1] http://googleonlinesecurity.blogspot.co.uk/2011/08/update-on... [2] http://src.chromium.org/viewvc/chrome/trunk/src/net/http/tra...

Google is actually quite diligent in this regard and have caught CAs with their pants down in the past.

They're taking it a step further and using certificate pinning in Chrome to catch MITM attacks in real time across a large portion of the internet. http://blog.chromium.org/2011/06/new-chromium-security-featu...

It's not scalable at all, but cuts out a large attack vector for a lot of communications. It wouldn't take a ton of pinned certificates to make a big dent in these NSA programs--really just look at the logos and make sure that each has their certificates pinned.

So let's assume the NSA still has these capabilities (a fairly reasonable assumption), and with SSL/HTTPS as a fairly feasible security option, how would these capabilities be possible? Either services aren't committed to and endorsing the use HTTPS/SSL and/or they are actively granting access to user information. Are those two reasonable conclusions?

I'm trying to understand why services are not taking a more active role in protecting their users' information if they are claiming to taking our privacy seriously.

To me, it comes down to being either incompetent or a liar, or both.

Did nobody look at the PowerPoint itself? From page 17 (http://www.theguardian.com/world/interactive/2013/jul/31/nsa...):

> * Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users.

> * These events are easily browsable in XKEYSCORE

As I understand it (and I may be wrong), most encrypted VPN traffic uses SSL. Given that XKeyscore data is only held for a few days (due to the immense volume) and given how nonchalantly they just throw out that they can decrypt VPN traffic, it sounds to me like they've either got the root SSL certs and are MITM'ing every connection they can or they've somehow broken SSL, either by breaking the actual encryption used or by exploiting vulnerabilities in how browsers handle it. If that's the case, then they don't need to ask Google or anyone else for your data, they can just read anything they want.

Poul-Henning Kamp: """With expenditures of this scale, there are a whole host of things one could buy to weaken encryption. I would contact providers of popular cloud and "whatever-as-service" providers and make them an offer they couldn't refuse: on all HTTPS connections out of the country, the symmetric key cannot be random; it must come from a dictionary of 100 million random-looking keys that I provide. The key from the other side? Slip that in there somewhere, and I can find it (encrypted in a Set-Cookie header?)."""

http://queue.acm.org/detail.cfm?id=2508864

That's an interesting idea.

Even better would be for the NSA to penetrate Thwate, Verisign etc and make the keys they "generate" non-random (perhaps only for a subset of certificates sold)

Yes, it seems like they have broken VPN or SSL in some way.

Perhaps they own or subsidize many of the cheaper VPN like has been rumoured for Private Internet Access? https://www.privateinternetaccess.com/

Uh, no. We aren't subsidized by the NSA or any part of any government or any organization or person for that matter. We bootstrapped Private Internet Access with 500$ and a lot of caffeine and have been profitable since our second month in operation.

We believe what the NSA is referring to when talking about "VPN startups" is the initial stages of PPTP sessions. PPTP has been crackable for a while, check out moxie's cloudcracker.com. We believe it highly unlikely that they have broken OpenVPN (which is what our application uses) or SSL.

Please see our stance on PRISM: https://www.privateinternetaccess.com/blog/2013/06/prism/

My question is, if they did this, why did each company have it's own join date in the prism timeline? wouldn't it just all happen at once? if they were tapping pipelines then the data would be collected regardless of the system, yet they all had very distinct dates that they were onboarded.

The PRISM program, which requires cooperation from service providers, is just one of the methods they use to collect user data. "Upstream" collection (beam splitters on fiber, etc) is separate and distinct from this, and yields a different set of data.

http://www.washingtonpost.com/business/economy/the-nsa-slide...

What makes you think they're even allowed to tell you the truth?

> are we still being lied to?

The answer will always be "yes".

From the slides http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

"Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users"

Does this mean using VPN is not very safe from dragnet?

If Google, Facebook, et al provide direct access to users' data, I'm fairly certain such critical infrastructure as VPN is also under NSA control.

I thought Google, Facebook etc. only had to provide access as they are compelled to under US law? If the VPN provider was non-US (and did not/claimed they did not keep logs of user activity), would this help? Or do you reckon the NSA has the ability to get at the data without the cooperation of the VPN provider? It would be great to have a couple of slides revealed on this area.

You probably don't need to break the encryption because eventually all traffic has to exit the VPN's company's endpoint, and at that point it can be captured. Meta data such as the browser's fingerprint can be used to tie traffic to an individual, for example, if you see them log in to a regular HTTP site with an email or a username, this information could probably be used to figure out who they are. Armed with this information, all other traffic originating from that endpoint (or elsewhere) with the same browser fingerprint.[-1] can be monitored. Weak keys can also cause the encrypted tunnel to be compromised. Also, PPTP is considered a very insecure tunneling protocol[0] but still used.

You could also break into the VPN company's servers and do interesting things too. There's also the possibility of timing attacks to determine the real IP address of the VPN user, although that's fairly a sophisticated method and quite difficult to do.

Bear in mind that this presentation dates back to 2008, which is a long time in tech years. Who knows what they're capable of now. All that's known is that they're not capable of less.

VPNs are useful for three things: protecting yourself against relatively unsophisticated bad guys sniffing traffic on a local network (for example, an unsecured wireless network), bypassing geographic content restrictions (e.g. using Pandora in Sweden), and circumventing ISP traffic shaping (often they'll not shape VPN traffic because it's used for businesses, and businesses can be whale customers).

[-1] http://en.wikipedia.org/wiki/Device_fingerprint

[0] http://blog.calyptix.com/2012/08/pptp-is-so-insecure-it-shou...

> all traffic has to exit the VPN's company's endpoint, and at that point it can be captured.

If the only thing they're dealing with is VPN's used as a private proxy for access to the public internet, you're right, and if so it's not so troubling (well, as in it is "only" just as troubling as having them access everyones web traffic).

But arguably most VPN traffic is exiting inside private networks and are intended for machines within those private networks. If they are capable of breaking or circumventing the crypto of those, then that's troubling at a whole different level because it potentially means massive unknown weaknesses in either specific crypto products, or in algorithms that have been assumed to still be reasonably safe.

The slide talks about VPN startups. Some corporate VPN connections could be also compromised for a number of reasons. There are possibly undisclosed weaknesses in the "gold standard" VPN solutions, such as OpenVPN, as well as the protocols they use.

Security's dirty secret is that security is an unobtainable goal. The goal of designing secure systems isn't to create something impenetrable (i.e. secure), but something that's almost impossible to penetrate. 100% secure systems are about as common as rooster eggs.

> VPN startups.

I took that to mean establishment of VPN connections, rather than companies operating VPN services.

Of course total security is impossible. But it would still be troubling if breaking common VPN services is not only possible but also doable with small enough resources that "any analyst" at NSA can just request it.

Many corporate VPNs are secured via RSA SecurID and their keyfobs. Several years ago the SecurID source was compromised by hackers[1] and it was suspected the master seed/key was lost. Imagine if the NSA had access to that key -- it'd certainly be a juicy target for them.

[1] http://dankaminsky.com/2011/06/09/securid/

The most reasonable assumption to make right now is to assume that the NSA does have the key. They may not, but but then again the recent revelations have been so absurdly horrifying that I wouldn't put it past them.

You often hear "What, I should go darknet? Are you crazy?!".

But that's actually the one thing that makes the most sense now:

https://en.wikipedia.org/wiki/Darknet_%28file_sharing%29

Yes and no. Remember, it's not a silver bullet or a secret security sauce.

That's true, but I believe, we need all the bullets we can get and use them in parallel. It's never about either-or.

I agree, and defense in depth is a core tenet of security after all!

There are not only correlation attacks as kintamanimatt describes, but "VPN" is a broad term that encapsulates a lot of old and broken protocols. They could be talking about breaking CHAP + PPTP.

>How do I find a strong-selector for a known target?

>How do I find a cell of terrorists that has no known connection to strong-selectors?

>Answer: Look for anomalous events

>E.g. Someone whose language is out of place for the region they are in

>Someone who is using encryption

>Someone searching the web for suspicious stuff

Lovely. Suspicious stuff and encryption. But wait! There's more!

>Show me all the VPN startups in country X, and give me the data so I can decrypt and discover the users.

Wait... what? I really hope that doesn't mean what it sounds like it means.

>Slide 22 [regarding determining who authored a Jihadist document] redacted.

Well that's interesting.

>Show me all the exploitable machines in country X.

That's cool. I'm guessing this is what Snowden meant by weak endpoint security.

>Over 300 terrorists captured using intelligence generated from X-KEYSCORE

>Slides 29 and 30 regarding this redacted.

What a shame.

I'd like to know where they are keeping these "over 300 terrorists" that have been captured due to this program.

How is one labeled a terrorist by this program I wonder?

By captured, I'm sure they mean captured via droning.

I'd like to know how they check for/track false positives.

This. This is what worries me more than anything. Where have these 300 people gone? There should be records, trials, something. 300 suspected terrorists going on trial over a period of 5 years should have resulted in huge wave of almost 24/7 publicity.

That the NSA is in the business of total surveillance is bad enough. But there is the faint hint that the NSA is in the business of making people disappear.

Black sites are still there. Black sites with dubious givernments do the disappearances for the CIA, wihout them getting their hands dirty unless they have to.

There's news about terrorist plots being foiled all the time. Why are you assuming there's no overlap? It's quite common in terrorist trials for some of the evidence to be sealed from the public (this isn't unconstitutional, as long as the defense gets to look at it).

And these days it seems you can be labeled a terrorist just for protesting..

Well, we know what happens when they're "caught" - they're put in Guantanamo forever.

Pretty fucking scary if you ask me.

>Someone whose language is out of place for the region they are in

Check (German browsing in English)

>Someone who is using encryption

Check (my PGP Key is on my website)

>Someone searching the web for suspicious stuff

Check (I'm a curious person, what did you expect?)

I guess I'm considered a terrorist nowadays.

Most of us here fit that description, which brings us back to the information overload argument: they are lowering their chances of success by increasing the ratio of false positives. Even disregarding moral and legal aspects, what they are doing is wrong because it's inefficient.

You're assuming that they see having false positives as an issue. You don't worry over such things when there's no problem with tossing suspects into a hole with no due process.

>> Show me all the exploitable machines in country X.

> That's cool. I'm guessing this is what Snowden meant by weak endpoint security.

That, plus things like Microsoft and Apple operating systems. Don't forget: it's proven they work with the NSA, so backdoors certainly are guaranteed (plus, with Microsoft, we also know they hand 0-day exploits over to the NSA before they're fixed, plus you benefit from all the viruses, trojans, etc.). Again, if you missed it, start migrating now: https://prism-break.org/

There was a document released by the Guardian a few weeks to a month back showing how they also monitor open source issue trackers to find exploitable flaws.

Oh I'm sure they're fighting on all fronts (which we will have to do as well...). But I do believe there are higher-hanging and lower-hanging fruits.

Slide 6 of the presentation clearly shows that pretty much every government is in on the program, with heavy concentration in western Europe.

One question, how did the dot in China get there?

http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

And look at slide 17:

"Show me all VPN startups in country X, and give me data so I can decrypt and discover users."

Holy crap. Is all encryption broken?

I find the "Show me all exploitable machines in country X" on page 24 pretty scary as well

There's no evidence that any encryption is broken (other than people misusing it (edit: or broken protocols like PPTP)). Anyone could do this kinda thing given enough motivation and money. Determining a VPN's users? Just monitor all inbound connections to the VPN service. Now you have the IPs of the users. The IP alone might be enough to know the user or a search on that IP might show them logging into other services that reveal their ID. Pretty simple.

I think so. Yes. IIRC, the first time I saw this leaked was a combo leak by a Navy Seal and a member of the Executive. The Seal leaked that they powered down Bin Laden's computers to take his hard drives after they shot him. The Executive member said that the drives were encrypted and it would take a few days to get the data. Jihadis are known to use a custom version of PGP with 2048bit RSA keys. They either used that, a COTS drive encryption program (unlikely), or reviewed and adapted an open source drive encryption program. In either of the likely scenarios they would have been using 2048bit RSA. Therefor, it is highly likely (due to the NSA having target motivation even if the drives weren't well encrypted) that the smooth barrier does not exist and the NSA can factor 2048bit RSA in a hours to days scale time frame.

Also, it was leaked that NSA TAO had a 70%+ success rate compromising Chinese systems. Even with the tech companies giving them secret zero days for an extended period of time, anyone that has been a blackhat knows they're not getting to a 70% success rate through exploits. Therefor, it's highly likely they can decrypt VPN/SSH (TLS) traffic encrypted with AES256/RC4-128/3DES and/or the RSA/EC public cryptography used. As you noted the leaked slide seems to indicate that.

Good info. Do you have any sources online you can point to?

Navy Seal Leak: _No_Easy_Day_ available from Amazon: http://www.amazon.com/No-Easy-Day-Firsthand-ebook/dp/B008MG1... (Notice how he makes no mention of ISA even being there, or doing any SSE).

I don't recall the source of the Executive comment. It was kind of buried in a news piece with a broad focus that I read. I'll look for it. Unfortunately, I can't recall the exact language to do a good search and find it. Sorry.

But, here they are saying they have it, it's encrypted, and they can get it in "weeks or months" (despite the large number of drives/filesystems with presumably different keys): http://www.cbsnews.com/8301-31727_162-20059825-10391695.html

The Executive and Legislature couldn't keep something secret to save their lives. And, JSOC leaks like a fucking sieve. If I can't find that particular leak on the web, I'm sure there will be another one soon with the same info. Every guy likes to talk to pretty news reporters and seem important.

Claimed 75%+ success rate attacking Chinese systems: http://www.scmp.com/news/hong-kong/article/1260306/edward-sn...

Asrar al-Mujahideen (the Jihadi PGP fork w/ 2048bit RSA): http://www.rbijou.com/2013/03/18/an-overview-of-jihadist-enc...

The news coverage sucks something awful. The thing is having enough knowledge of mil/ir/tech/math to put all the leaks together.

Breaking RSA is just a matter of managing to factor prime numbers faster than anyone else, isn't it? Unless if there is some sort of oversight inside the RSA algorithm that allows the encryption to be broken easier.

Do you have more information on the smooth barrier? I did a quick google but didn't see much relevant.

As the size of a semiprime increases, the number of smooth numbers that can be discovered (the "yield") by the GNFS with polynomials selected with academically known optimal polynomial selection algorithms decreases. With a reduction in smooth candidates the GNFS sieve operation can be wholly unsuccessful. If a smooth barrier exists (such as a semiprime size where smooth yield becomes deficient) factoring time degenerates from the GNFS improved rate to old school factoring rates due to the need to pivot. Yield decay has been observed <2048bit. If 2048bit is easily factorable for the NSA, no barrier challenge is suggested.

Or, they have a better (probably non-Ring) factoring method. I can't conceptualize what that might be, though.

I'm not sure it's relevant whether the b-smooth barrier exists of not, since that assume use of NFS.

There's a reason the NSA is pushing folks to use Suite B ciphers including Elliptic Curve along specific curves. It's not unreasonable to think that the NSA mathematicians have proven some relationship between EC and prime number theory in general.

There is some public domain work on this topic. See [https://en.wikipedia.org/wiki/Lenstra_elliptic_curve_factori...].

This might help explain in part the NSA's desire for large memory vector supercomputers going back to the 1990s over distributed memory MP systems.

Interesting comment! Yes, I have been trying to avoid EC because some of the random walk stuff I read made me uncomfortable given standardized curves. I always thought that NSA vector register desire was strictly due to block size of ciphers (particularly Russian). This was definitely true when DES/3DES where in use. Then again, I thought Bluffdale was just to crack old Russian intercepts with GPU like custom hardware. BTW, a Cray hw engineer and I talked about how Cray was trying to pivot into Bioinformatics since the gov biz was no longer robust (in 2004, IIRC?).

The whole reason the USG rescued Cray in the late-1990s/early-2000s was to insure the continued availability of large memory image vector supercomputers. Part of this may have been to it being less costly than converting their processing systems from vector codes and algorithms to massively parallel distributed processing ones. At that time the cluster interconnects were much, much slower in terms of both bandwidth and latency than they are today. Solving very large sparse matrices would have been tougher on an MPP system than on a vector one. You can read about some of this history in Bamford's "Shadow Factory."

There have been a number of very cost effective hardware approaches proposed for significant acceleration of both the sieving and linear algebra components of the NFS. Many of these proposals could successfully and cost effectively attack a 1024-bit number in the 2003/2004 era. The process at that time was around 130-nm. Today's process would have features at the 32-nm or 22-nm size. Today there has been a 100-fold increase in performance since 2003. (See http://tau.ac.il/~tromer/cryptodev/ for an overview.)

Combine this specialized hardware with an algorithmic improvement that gets to O(log n) or O(n log n)....

AES appears fine. The NSA and USG in general make a very strong effort in the 2000s to move all civilian command and control systems for satellites to AES-256 with TRANSEC capabilities. A brute force attack on AES-256 with a quantum computer should be on the order of 2^128 operations with currently know QC factoring algorithms. AES-128 looks weak at 2^64.

If the NSA can break something, they need to assume that their primary opponents can do so or will do so soon. China specifically comes to mind here. The can not release cryptography suites with known vulnerabilities. It is widely thought that it is more importantly to secure one's own signals before intercepting and decrypting one's enemies.

I think everything on the internet needs to be moved to Suite B protocols with forward secrecy enabled. AES-GCM overcomes all the known attacks (i.e. CRIME) against AES-CBC and AES-CTR.

I get the impression that the NSA is eight to ten years ahead of the public domain cryptographers in some areas. I think this gap is shrinking slowly. However, I have also heard that the NSA is preventing publication of some papers developed in the public domain due to national security reasons.

I know nothing about this stuff, so apologies for my naiveté, but what technical barriers prevent us from changing from 2048 bit encryption to something of a much much greater magnitude? 2,048,000 bit (or whatever).

RNG/PRNG output quality and the internals of the algorithms.

I just noticed that the presentation is from 2008. Imagine what capabilities they've added in the last five years.

And what about all the dots along Antarctica?

The only thing I can think of is satellites?

Not to mention all those Antarcticans!

Could be an embassy.

I wonder how Sencha (http://www.sencha.com/) feels about how the NSA is clearly using their ExtJS framework given the screenshots.

I guess this kind of puts different perspective to the whole debate that came from JSMin's "The Software shall be used for Good, not Evil." clause (http://wonko.com/post/jsmin-isnt-welcome-on-google-code) given that conceivably your open source framework might be a significant part of something like this.

Its also very likely the machines are being run on Linux boxes. Should Linus being losing sleep knowing he aided the NSA in this? Even the very database this system runs on may be an offshoot of the Google BigTable paper. Should Google have never opens sourced the software that eventually became HBase/Hadoop/Cassandra because of the NSA?

A tool is a tool. I don't think Henry Ford should feel guilty for enabling people to kidnap children with greater speed.

No it doesn't. This issue was raised and thought out long before JSMin, for example in the case of the license that forbade use by the South African military. It's a bad idea.

ExtJS is widely used on government systems and has been for years. If you want to deploy a rich web application that can handle large data tables with infinite scroll, filtering, sorting, etc., and run it all on IE >= 6 ExtJS is your only feasible choice.

Snowden deserves the Nobel Peace Prize and the Vatican should consider canonizing him.

To be canonized you have to have performed a miracle, but If he somehow get's pardoned by the DoJ or the Obama administration we could probably consider that requirement met.

You've performed a miracle, yourself. A good joke in the comments section of an HN post! :p

At this rate if the surveillance programs get shut down and most of the Patriot act etc get rolled back, it would be a miracle.

Imagine if storage limitations weren't holding back the NSA.

Those 60TB density HAMR[1] drives that are due in 2016 are really going to take invasive to a whole new level.

[1] http://storageeffect.media.seagate.com/files/2012/03/perpham...

20 TB per day is nothing these days.

Imagine if the American people weren't holding back the NSA.

Oh, wait...

mmh... but maybe the military is also ahead in that area? Maybe DARPA has 1TB GB/s drives?

I don't quite know what you intend "1 TB GB/s drives" to mean.

But note that you can buy off the shelf PCIe cards with SSD's mounted that will give you 1TB storage and an aggregate read bandwidth of more than 1GB/sec today. I've got three sitting in various servers. They're expensive, and frankly for the future I'll rather get a couple of extra SATA III controllers and get multiple "regular" SSDs on separate controllers for that reason, but they're available.

For NSA style data collection, though, the collection is trivially to do in parallel: Hash all keys to a "virtual bucket", and hold a map of virtual buckets to physical servers. Then when you want more capacity, you add some physical servers, reassigns some of the virtual buckets from other physical servers to the new ones, and synchronises any old data (given that NSA claims they could only hold the full data stream for three days, you don't even need the hassle of moving data, just make collection on different days map to different virtual buckets, so that on day one you "just" reassign virtual buckets the content of which is being expired on the old servers anyway, on day two, the next set etc. - you maintain full spread of read/write traffic by ensuring that in normal operation all servers have an even spread of "day 1", "day 2" and "day 3" buckets).

It's amusing they see storage as an issue, but of course this was in 2008. Today I have 6TB in my home NAS, and my perfectly off the shelf tower case can easily fit 40TB+ with current size harddisks (though I doubt the noise would make me popular at home).

Who said that they were using spinning storage? You can store an insane amount of data for your dollar if you are willing t use tape.

For example, IBM have a robot tape library that can store 900PB[1], Quantum can fit ~5PB per full rack equiv[2], and there are many many more.

[1] http://www-03.ibm.com/systems/storage/tape/ts3500/index.html

[2] http://www.quantum.com/products/tapelibraries/scalari6000/in...

Another data point on the relationship between government and terrorism:

I live in Columbia, South Carolina. A mile from my house there is a prominent statue of Ben Tillman. Tillman was an explicit advocate of terrorism, and indeed personally engaged in it [1], which drove his popularity and ensured his election to the governorship and the United States Senate.

Government programs such as the NSA's exist to protect the interests of the powerful. Same as it ever was.

http://en.wikipedia.org/wiki/Benjamin_Tillman

Holy shit.. they did really write a GUI in Visual Basic to track the killer's IP

Thanks, there isn't much funny about any of this, but that got a legitimate laugh out of me. For anyone who doesn't know the reference, a clip from CSI:

http://www.youtube.com/watch?v=hkDD03yeLnU

I think the guardian got trolled hard. This is too ridiculous to be true.

I think you're in denial.

>Foreignness factor

I know NSA's mandate is to spy on foreigners, but it's still very jingoistic and xenophobic that not being American makes it OK to spy on you.

One could assume that Americans are spied on by foreign governments and the data is just exchanged. The US spies on Brits, the UK spies on US persons, and the both compare notes.

This is exactly why "every country spies on foreigners" is unacceptable. There's no reason to assume that this "note-sharing isn't already taking place.

It's less of an issue since many communication and internet services are located in the US.

Actually this is exactly what occurs. Intelligence exchange among America and its allies under Echelon, ANZUS, and UKUSA have been used in this exact way to end-run around anti-domestic surveillance laws.

A 'fun' bit of weasel-wording by the chairman of the House intelligence committee: "He's lying. It's impossible for him to do what he was saying he could do." They seem to be denying it, but all they're really denying is that Snowden had access to the system personally.

It's impossible for him to do it, he's in Russia!

They mean he didn't have the "capability" to "collect" that data, which in NSA newspeak means he had the technical means, but not the legal authorization to do so.

Hey folks,

Just wanted to add a note and say that if you're angry about this, the best thing that you can do is to get out into the streets and protest everything that's been going on. Check out the Restore the Fourth rallies happening this weekend, share them on social media, and sign up for your local event.

http://1984Day.com

Getting out into the streets is the single most significant thing you can do - even more effective than calling your legislators. The events on Sunday need to be bigger than the events July 4th for this to really be a success.

Noticed one of the screenshots have a URL. It's a little blurry, but I suppose it's an intranet URL since the TLD looks like .nsa

URL looks like: https://gamut-wakefield.ein.nsa/utt/UTT/do/FRNewSelector#sel...

Interesting addition. I wondered what some of these abbreviations in the URL stood for. Upon searching, I found this:

http://www.techcareers.com/job.asp?id=64332188&aff=C014D02C-...

Job posting, requiring top-secret clearance, looking for people that have experience using certain tools including "GAMUT/UTT" - notice the URL from the NSA doc has "gamut" and "UTT". So i further looked into GAMUT/UTT and found this:

http://williamaarkin.wordpress.com/2012/03/13/nsa-code-names...

This is the most terrifying thing i've ever seen. I'm not exaggerating in the slightest.

One of the screen shots:

http://static.guim.co.uk/sys-images/Guardian/Pix/audio/video...

says: Top Secret Comm(?) REL() to USA, AUS, CAN, GBR, NZL

confirming the previous suspicions that many other governments are on board.

Der Spiegel actually has reported a few weeks back about XKeyscore [1] and that it is used by the BND (Germany's NSA). I.e. all this data is also available to the NSA equivalents of Australia, Candana, Great Britain and New Zealand.

Many Americans trust their government (unfortunately), will they also trust the other governments?

[1]:

http://www.spiegel.de/international/world/german-intelligenc...

http://www.spiegel.de/international/germany/german-intellige...

Not GER, it's GBR as in Great Britain

http://en.wikipedia.org/wiki/UKUSA_Agreement

I'm pretty sure that's actually GBR - Great Britain.

It definitely is; look at the actual (hi-res) presentation: http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

Those are the Five Eyes countries: http://en.wikipedia.org/wiki/Five_Eyes

Good catch -- and really, I find it to be quite foreboding in terms of how indomitable it is precisely because of the secrecy of the program.

"This was a secret treaty, allegedly so secret that it was kept secret from the Australian Prime Ministers until 1973."

This is indeed a trend, and I speculate that NSA (and NSA-like entities in the other 4 eyes/countries) probably communicate information and abilities to prime ministers and presidents of the respective countries very selectively.

Bonus: The NSA likely can get around the "no spying on US citizens" by just requesting data from those governments, who proceed to pull it out of the NSA's web interface.

That's COMINT, or Communications Intelligence, basically the type of intelligence that XKeyscore is part of. It might say HUMINT if the intelligence was collected from human sources.

REL TO likely means release to.

As I've said before, the realisation that most countries do this sort of thing comes as no surprise.

Livestream to senate hearings covering all this can be found here: http://www.judiciary.senate.gov/hearings/hearing.cfm?id=0d93...

As of this moment it's all about FISA. Wonder if this new allegation will be talked about.

The senate hearings are with regard to phone metadata. This is a totally different show.

they did mention it offhandedly, the woman(i was only listening) started making a list of things that the nsa should release yearly including how many crimes 702 and whatever helped prevent, and the main guy asked her to add a view, in the light of the news posted today, but said they would investigate that further at a later time. sorry it's poorly detailed but just wanted you to know it was mentioned.

You have to admit these guys are working on some cool problems. If you don't have a problem with the legality of it or potential for misuse it looks like a really interesting place to work.

Yeah and it looks like intelligence & skills doesn't always equate with having morals.

That's exactly how they get people to work on it in the first place. If you have no conscience there are lots of places where you can work on 'cool problems'.

Weaponizing a nuclear reaction? Cool, right?

I'm going to give you the benefit of the doubt and assume you're not condoning their actions, but in case you need a more clear example of where this paradigm breaks down:

IBM & Nazi Germany, Nazi Germany in general, etc. Ironically, sillicon valley came into existence building military SIGINT/ELINT systems for the cold war.[0]

[0] www.youtube.com/watch?v=hFSPHfZQpIQ

I think there is a developing consensus that the emerging US police state (and that of its allies) aren't appropriate for constitutional democracies. Comparing them to Nazi experiments or nuclear weapons is still a bit premature. The potential is there for it to turn really ugly which is why the time for political action is now. But national security is important and worthwhile within constitutional limits and under democratic oversight and I suspect a scaled back NSA would be no more evil or less interesting than Bletchley Park or the development of Radar.

Totally agree with everything you've said. I wasn't trying to equate helping the NSA build spy tools with building equipment for the Nazis -- just how doing so can be a slippery slope because it scales to such awful things as well.

I guess I felt the need to comment because it sounded like you were saying, "Hey, I get why they're doing it. Sounds like it'd be fun!", and I feel like having that attitude (even if I trust someone like you to know to stop before things get truly out of control) is dangerous.

As were certain medical facilities in WW2.

I just read the actual XKeyscore slide deck. Unlike a lot of leaks, these slides are totally worth a look-see: http://www.theguardian.com/world/interactive/2013/jul/31/nsa...

A couple years ago there was an AMA on Reddit from someone saying he was very deeply involved in spying on the general public's online lives, "at a level you can't imagine". Many technical questions were asked, all answered properly. I could never get it out of my head and now that Snowden has emerged I can't stop thinking he was the OP. Wish I could find this AMA again.

Do you happen to have a link to this AMA? Not sure what to search for to find it.

I can't find it, can't remember anything to search by. Very frustrating.

This has been up here for 5 hours and on the Guardian's website for nearly 6 hours. How is it possible that not the NYTimes, FOX, NPR, the Washington Post, or CNN have picked this up? These organizations are an embarrassment to the profession of journalism.

https://news.ycombinator.com/item?id=6136016

Here is the newsdiff of today's NYT article on the disclosure of the declassified docs. [1] Talk about burying...

http://newsdiffs.org/diff/290704/290768/www.nytimes.com/2013...

So rather than making a headline, update an article on another topic, specifically update an article about a PR move by the White House intended to stop talk on this issue. Assholes.

I don't think its so much "an embarrassment to the profession of journalism" in terms of incompetency but rather willful ignorance.

Greenwald said in the comments, there is a lot more to come:

> That House vote was about one specific topic - bulk collection of phone records - that this newest article has nothing to do with. That House vote isn't the be all and end all: it's just one small battle in what I can assure you will be a sustained and ongoing discussion/controversy.

> There is a lot more to report still. Accuracy is the number one priority. That takes time.

Here's an interesting legal question:

If a non-US resident or NSA target posts a thread on HN, and a US person replies to the thread, is the US person now open to unlimited data collection?

Alternately, if you Facebook-like the same thing an NSA target has, are you then subject to unlimited data collection?

I don't know about forums, but I suspect that being Facebook friends with a non-US citizen is enough to subject a US citizen to unlimited collection.

The information we have already shows that the US person/non-US person distinction is purely cosmetic, meant to allow them to pretend that they're at least trying to respect US laws to some tiny extent. It's rhetorical.

In reality you are always a valid target, US citizen or not.

I don't think I have been more conflicted about this. I've just been talking to my cofounders about the technical feasibilities of XKeyScore, and honestly, our back-of-napkin engineering configurations indicate this is really an awesome project to be working on.

On the other hand, this is categorically 'evil' by my and my cofounders' ethical standards, and really, no one is safe. And that bugs the hell out of me.

On the one hand: really fucking cool. On the other, I really do not like the idea that I am being spied on.

I'm not sure how to process this information.

Welcome to working in the modern advertising industry.

I AM currently in the advertising industry

No wonder you think it's cool.

Sigh

> On the one hand: really fucking cool.

I just don't see how this could be considered "cool". There are plenty of other marvels of modern computing that aren't so sinister.

I think a better word for this is "scary", due to the level of cooperation from corporations and the level of secrecy it was running under for so long.

I think it's insane that so many people are pointing the finger at Snowden, yet no one is pointing the finger back at the NSA / US Government.

The scale, depth and technical sophistication of everything I've heard and read so far has made me change my mind on whether or not there is a technical solution to NSA and GCHQ surveillance. I'm now convinced that the only way to solve this is through politics. We need representatives that will enforce our rights to privacy, not clever hacks.

That said, we need to do the technical solutions as well. Don't use the difficulty as an excuse to give up on one partial solution, when the other solution is partial as well.

Ok, so ignoring all moral/ethical issues with this. Wouldn't it just be awesome to work on a project like this? Unheard of funds, tons of data, interesting CS problems all around. I am sure they did everything possible to make it miserable on the developers but nonetheless... sounds fun from a completely detached CS perspective.

Yes, I was thinking the same. Sort of the software engineer equivalent of flying the best fighter jets and recognizing that you're going to be using them to bomb people in mud huts instead of dogfighting equally-classed opponents.

Summary of how this story is being covered around the world vs. in the US.

http://imgur.com/a/tS7h4

i was shocked by this too. about an hour ago i went to CNN.com and searched the keyword xkeyscore and got 0 results. Not a single article?

The map on page 6 is interesting. Server locations of note:

  Moscow, Russia

  Caracas, Venezuela

  Tripoli, Libya

  Hubei Province, China

  Burma

  Lagos, Nigeria

  Saudi Arabia

  Iran (and geograhically surrounding Iran)

  Ukraine

Based on page 13, I wonder if Google have any servers at these locations?

Oh, what a surprise: http://royal.pingdom.com/2008/04/11/map-of-all-google-data-c...

Does that look familiar?

I'm very suspicious of Google's role in all this, but there's not much overlap in the list above and the map you refer to, outside of the normal population and business centers in Europe and the US. I didn't see any in Google data centers in Libya, Burma, Nigeria, Ukraine, Saudi Arabia, Iran, or even Venezuela (the only one in S. American looks like it's in Brazil). Based on a quick glance, the only overlap between the list above and Google's server looks like Moscow, a major global business center.

Were we looking at the same map?

Sorry, I wasn't suggesting Google were complicit, but the NSA need to suck up search queries, so it makes sense to locate these NSA servers right next to local Google installations.

Again, page 13; a local Google Pakistan search query.

The thing that blows my mind, is you hear over and over again about Billions of dollars being spent on large software projects for the government that seem fairly simplistic that ultimately fail.

The NSA is accomplishing some pretty impressive things, what are they doing differently?

Spending tons of our money while being accountable only to themselves.

Spending tons of money didn't help when DoD was trying to field DIHMRS. The military still runs on an ancient COBOL-era payroll system because they can't successfully develop and field a replacement, despite some billions in wasted $$$.

Hiring mathematicians and computer scientists instead of MBAs and public policy people.

Probably also not having to follow government contracting rules (lowest bidder, preferring minorities and veterans) because who would have the authority to review their purchases?

Siphoning money away from "legitimate" projects leading to their ultimate "failure". It's just shell game accounting.

Slide 13 just got slightly redacted.

http://imgur.com/a/SVerP#0

Because it has someone's personal data in it, and the Guardian respects privacy - unlike the NSA, who use it in presentations. That's the only reason they've redacted anything, if you look at where the slides sit in context.

The 64. IP address on that slide is to Google (obviously), here's the other end:

GeoIP City Edition, Rev 1: PK, 08, Islamabad, N/A, 33.700001, 73.166702, 0, 0

5.157.65.58.in-addr.arpa domain name pointer mbl-65-157-5.dsl.net.pk.

23674 | 58.65.157.0/24 | MBL-AS | PK | NAYATEL.PK | MICRONET BROADBAND (PVT) LTD.

Wow. Just Wow.

For years all of this was in the back of my mind as being capable but my not wanting to think like a conspiracy crackpot just dismissed the thought as it couldn't be possible. A conspiracy takes a lot of co-operation from within large corporations who must also remain it a secret. Surely someone would have a conscious and leak it? Or one of companies we all look up to as a modern example of do-good company would say "Hell NO" to the attempt and then let the world know what was attempted. Guess that was eventually proven true with Snowden (a real hero imo), just shocked they were able to operate to the scale they did for so long before a Snowden came along.

In my mind, this is not so much a shock to me regarding the NSA as well as the current evil government we have had in place. Doesn't take a genius to realize the president lies to our face on TV about trivial issues/promises, so expected for top secret stuff.

What is the BIG stomach churning shock to me is the very companies that we have come to know that are multi-billion dollar conglomerates providing service/products for millions for every day use has been a part of it. A part of this secret web while all the while proclaiming privacy for it users. I guess at end of day profits still rule the roost. "Just do this for us, turn a blind eye, and you get to go on making your billions". I wonder how many CEO's knew of all this. Gates? Zuckerburg? Etc etc.

I feel like I have no outs now. There are no alternatives to current establishment of companies that make our lives easier. Should we all wipe our PC's and use Linux, sell our phones and use Ubuntu Phone, not pay for SSL certs anymore (another mafia), etc?

I wrote an e-mail to Congressman Mike Rogers about his misleading quote in this article. I encourage others to reuse my template and also ask him to justify his misleading remarks about Snowden's statement: https://news.ycombinator.com/item?id=6134672

We should start hold our public servant to task for lying to the American people about these programs.

I just censored an email I was about to write in case it is used to discredit me in the future. Jesus. The thought police are here :-(

At what point do the mathematical limits of data mining kick in here? How useful is all this information?

I'm not an expert in this area of mathematics, so I could be wrong, but my impression is that as the haystack becomes larger the problem of false positives becomes more and more severe.

As a data miner, what you want is the maximum number of "hits" (of whatever you're trying to hit) with the minimum number of misses and the minimum number of false positives. My impression is that this becomes progressively harder-- the golden region between too many false positives and too many false negatives becomes smaller and smaller and harder to hit.

Eventually you either miss important hits, namely the next terrorist attack, or you get swamped with false positives that you have to manually investigate and rule out.

I'd love someone who does know more here to chip in, but my personal suspicion is that this actually has a pretty huge pork angle to it. How much money are the contractors getting for building this stuff?

From the article

The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours."

so let's say there's a law that says "any American company doing business with a company that does business with a known terrorist organization will have a bad day"

you don't need to use some kind of fancy data mining algorithm for this to work (generating false positives), you just need a ho-hum graph traversal algorithm and unbelievable amounts of graph data to generate "candidates for investigation".

US Company A -> intermediate 1 -> known terrorist group B

US Company A -> intermediate 2 -> known terrorist group B

US Company A -> intermediate 3 -> known terrorist group B

Each set of links is just one lead to investigate, but having a giant graph to work off of would make generating those leads simply. You might find out that intermediate 1 is a local falafel delivery place that "US Company A" uses for lunch catering. Can probably strike that one off the list. intermediate 2 is a utility (no choice but to use the local water monopoly), but intermediate 3 is a material supplier that employs several low level delivery guys from known terrorist group B, and the founder of the company is a cousin of the founder of known terrorist group B.

So I'd wager it's not as simple as just running an algorithm and automatically sending out Skynet drones to blow things up. There's some kind of more subtle assessment being made, with the systems just providing help to the analysts.

Fitting/sinister that the top of their stored data pyramid is titled "TrafficThief"

That UI looks awfully similar to a theme I've seen used in SharePoint Portal Server. I hope that's not what they use for the front end, but I wouldn't put it past them.

Having worked with SharePoint extensively for years, I'm highly confident this isn't the case. It's just plain-old crappy custom-coded HTML4 forms from what I can tell and similar colors, so one could easily think so. But at closer look, it definitely not any of the SP versions 2003, 2007, 2010 or 2013. Maybe 2001 but I highly doubt even that.

Pretty sure it isn't - some of it looks like old ExtJS if anything.

The more worrying thing is that they're still apparently using IE6. [EDIT: OK the presentation is from 2008 but still!]

This, to me, is like showing the alleged UI on an Apple IIc:

It just about proves that it's not true, to me.

And, according to Greenwald, there's a lot more to come. From the comments:

"There are thousands upon thousands of documents and they take time to read, process, vet, and report. These are very complex matters..... there is a lot more to report still. Accuracy is the number one priority. That takes time."

Just a friendly reminder that if you looked at the slides, you have read a classified document, and therefore are guilty of a Federal felony. Cheers!

And the NSA is passing your details on to the FBI riiiight about now.

Enjoy your federally-funded vacation!

At least you can be sure that dinner will always be served.

I don't think it's a felony unless you have signed a government agreement, like working as an NSA contractor.

The Guardian strongly implies this system is used to intentionally target US citizens in violation of the law, but then admits that would be "illegal." I wonder if the leaked presentation touches on this point.

The Guardian doesn't 'admit' anything (it wasn't hiding anything in the first place), and legality doesn't predict whether actions are being taken or not.

>I wonder if the leaked presentation touches on this point.

That seems unlikely to me, as this is a technical presentation.

Are the major news networks ignoring this story? Briefly checking, I only see Fox News reporting related stories, naturally blaming the Obama Administration (perhaps fairly in this case).

"The NSA documents assert that by 2008, 300 terrorists had been captured using intelligence from XKeyscore."

So, even IF this number is not just another lie, XKeyscore has been made worthless, with something ridiculously small as the 2 prison breaks of the recent days.

That means: What remains is a police state that is not even "secure".

Good job, governments/lobbyists/"defense" corporations.

Germany used NSA's XKeyScore spying software: magazine report http://www.globalpost.com/dispatch/news/regions/europe/germa...

It seems this was meant to be declassified in 2032.. I guess by then they were hoping this would be so institutionalized and pervasive as to be the norm.

Also I wonder to what extent this is really used to hunt terrorists down and how much of it is used to gain political or economic advantages over other countries.

Secrecy is like copyright - keeps on getting extended, or the documents just get "mislaid" long before the declassification date.

I realize that it isn't morally right, but I think such data storage is inevitable. With the rise of instant communications, the amounts of data people generate are massive, and old school law enforcement can't keep up. Thus with the increase in technology, there's going to be an increase in counter-technology.

I guess what we need to ask ourselves now is whether we want any secrets at all. A true Panopticon -- a society where everyone could see what everyone else was doing -- might bring a "freedom" from certain types of subterfuge, and attack.

Then again, I don't want to live in it.

That leads us to the question of how we handle the flood of data when looking for hostile activity, because governments are certain to use available technology to trap, parse and search that flood.

Imagine what Nixon would have got up to with this capability. He would probably still be in power!

Makes me Wonder, if the Internet in this widespread form, was allowed so that they can snoop (so easily)?

When I was a kid, my father, had told me a story that in Russia people are scared to speak their minds, for fear of being snooped via any hidden gadgets in the walls.

And it's not even limited by the internet. NSA collects every piece of information they can get their hands on, whether it's data on the internet or any other network, or spectrum signals, or simple imagery. And they do that by every means possible. James Bamford wrote years ago a number of books on the subject. And even before him, David Kahn painted a pretty clear picture. Why is everybody so excited so suddenly? Is there somebody on this forum who believes for a split second that Mr Obama and/or the US House/Senate are prepared to lay off way over 300,000 intelligence community workers, contractors and what have you??? -RTF

Declassify on January 08, 2032. I wonder what kind of reaction the people of 2032 ( we who are still living that time ) would have had if they found out. Would they care? Worse reaction? Probably be used to gov. spying? It's a scary world.

I asked this in a deeper thread, but i would like to reask anyone that can explain. If the NSA is tapping pipelines as it seems they are, wouldn't the sources such as facebook and google all come online at the same time? if they were in fact referring to the pipeline access as their way into facebook and company, why did they all have different onboarding times? wouldn't they have all come on at the same time: the time when they tapped the pipelines? Maybe i misunderstand the process. I get that maybe they had to write some interface that interpreted the packets and sorted them as such, but that wouldn't take years.

My interpretation is that the NSA basically have two main forms of collection: data directly from fibre intercepts, and data obtained (via voluntary agreement, court order, or otherwise) from private companies. This slide [1] would certainly suggest such an arrangement.

The fibre intercepts would fairly easily give access to HTTP traffic, and Facebook/Google/etc. would probably 'come online' at about the same time (there will likely be some differences as it appears there is a need to code a plug-in/processing engine for each major source to pull out usernames etc.[2])

What exactly the dates in the PRISM slide mean is somewhat unclear without more information. It could be, for example the date that the first court order is made, or the date when the company provides to the NSA a more automated way to query the data. I doubt that those dates are related to the fibre intercepts though.

[1] https://image.guim.co.uk/sys-images/Guardian/Pix/pictures/20...

[2] https://image.guim.co.uk/sys-images/Guardian/Pix/audio/video...

so if that's the case, then it would seem that facebook, google, etc. are still lying? or could this be more gag order stuff?

No it would not... At this point, it seems quite feasible that Facebook, Google, et al., are telling the truth at least in some respect. The point is that there are TWO different forms of data collection: PRISM and XKeyscore. PRISM happens via court order and is what we've heard about primarily up until today. XKeyscore is a separate program and does not require the compliance of Google, Facebook, et al. In fact, it doesn't even require their knowledge.

The key now is to see who exactly is letting the NSA tap their network hubs to sniff the entire Internet. These will be your Internet Service Providers...

The ISPs might not have a choice either, thanks to CALEA.

Definitely. Thanks for pointing that out! I didn't mean to imply either way, but looking back, I did use some unfortunate phrasing.

Slide 23: "Show me all the Microsoft Excel spreadsheets containing MAC addresses coming out of Iraq so I can perform network mapping"

Does MS Excel store your MAC address in the xlsx file?

Um... surely this has to be a spoof. "Select foreignness factor"?? really?

The user interface and way this is done just seems to amateur hour to believe this is actually true

The numbers seem way off and too keystone cop to be true. 20 terabytes is not large for the NSA.

It can search BCC?? Only the sender has them. so everything would have to be collected at each ISP (which isn't impossible).. but I think the guardian has been trolled.

I'm calling this a troll.

If you or I was in charge of the NSA there would not be a visual basic interface or a select foreignness dropdown. I'm calling bullshit on this.

Wait, are you talking and agreeing with yourself?

In the PRISM related documents they used that exact term, "foreignness".

http://www.theatlantic.com/technology/archive/2013/06/bombsh...

If you're using Gmail then Google sends your email, so Google knows your BCC lines.

Keep in mind that this was from 2008.

That's 20TB per site. Who knows how many collection sites they have?

Says in the full slides on guardian over 500 sites (this was 2008), I wonder how many they have now..

Umm ... hello? Defence contractors anybody?

In one sense, it is amateur hour ... but if you buy enough hours from enough amateurs ...

As one slide indicates, the ability to search HTTP activity by keyword permits the analyst access to what the NSA calls "nearly everything a typical user does on the internet".

It seems like everyone's been attacking the wrong folks. From this article it appears that bulk of the data is being tapped at the data center level and then parsed. This begs the question how it would be able to make sense of https traffic.

People aren't going to care about this until they understand what consequences this may have to THEIR personal life. Live with it. Not changing unless the knowledgeable/wise starts educating the general population on how they effects them.

Most people might speak against it (include people here) but at the end, they have the "I'm not doing anything wrong, who cares, not worth the effort" mentality.

I am curious: Suppose this is true and NSA analysts have the technical capability to access enormous amount of information with no authorization. But if they do do that, agains the law and the rules and their actions are recorded in the system, they could face penalties no? I mean I could kill someone with a hammer technically, that doesn't make hammer bad per se, does it ?

It says can be audited. I suspect the number of times that has been done is vanishingly small, perhaps only on Snowden, after his revelations :) It smells to me like plausible deniability. Claim you have set up a system where things can be audited, don't ever answer how frequently it is done or what the penalty is for abuse of the system. Top secret and all. This kind of system really should freak people out. I imagine it would be a great blackmail info system if people start speaking against the govt or "causing problems". I can imagine public figures don't want to be publicly shamed for their online activities, gambling, porn, affairs, etc...

I wonder how they store all that. Surely a side benefit of this could be NSA contributions to CS journals about database techniques.

Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports? I think it's more likely this claim is made simply to generate FUD in the general population.

Hadoop[0] and OpenStack[1]

I think the era of government being far ahead of commercial tech capability is over. The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors.

[0] http://online.wsj.com/article/SB1000142412788732349560457853...

[1] http://www.youtube.com/watch?v=NgahKksMZis

The government mostly outsources now (a problem Snowden identified in terms of information control) or develops in-house with vendors.

There are a lot of fingers in that pie. Oracle, for example, has a National Security Group, whose job is to come up with "solutions" and then try to sell them to three-letter-agencies.

>Also I doubt the veracity of the claim that they collect "nearly everything". Wouldn't they show up on, say, Sandvine's Internet traffic reports?

Why would they? They deal with ISPs, backbones and such.

Another benefit could be realised in the future if historians and linguists manage to get access to all this data. I imagine that researchers of the social sciences would end up enjoying the same sort of large-scale collaborative projects that particle physicists or genomic bioinformaticians currently have with their huge datasets.

> I wonder how they store all that.

Well, now they have a massive data center in Utah. That's most likely where it's all going today. Standard open source tools are all you really need.

> Wouldn't they show up on, say, Sandvine's Internet traffic reports?

No. If some script kiddie/hacker type installs a packet sniffer and logs all your traffic to your ISP, that won't show up anywhere. Traffic goes somewhere. You're sending packets out. Merely logging packets is entirely passive and undetectable.

I am going to be obliterated for this comment! Does the fact they've caught 300 terrorists in anyway justify what they're doing? I am not saying it does, I just wondered what people's thoughts were (although I can guess!). It's interesting that it was included in the article in an attempt to give it some 'balance'..

Sir Thomas More decides that he would rather die than lie or betray his faith. And one moment he is arguing with the particularly vicious which hunting prosecutor. A servant of the king and a hungry and ambitious man.

And More says: “You’d break the law to punish the devil, wouldn’t you?”

The prosecutor says: “break it? I’d cut down every law in England if that would take it to catch him”.

“Yes you would, wouldn’t you?” And then “When you would have cornered the devil and the devil would turn around to meet you, where would you run for protection, all the laws of England having been cut down and flattened? Who would protect you then?”

Every time you violate – or propose the violate – the right to free speech of someone else, you in potentia you’re making a rod for your own back. Because (…), to who do you reward the right to decide which speech is harmful, or who is the harmful speaker? Or to determine in advance what are the harmful consequences going to be, that we know enough about in advance to prevent? To whom would you give this job? To whom you’re going to award the task of being the censor?

http://howtoplayalone.wordpress.com/hitchens-on-free-speech/

These are quotes from the Bolt drama "A Man for All Seasons," not Sir Thomas More himself.

Guess it was one of those days where Hitchens had a drink too many and got his quotes mixed up.

More likely he assumed the reader knew what he was talking about.

I would assume the same, yes.

In my opinion, no, but you do raise a question that legitimately is in the realm of politics: how much free reign should the secret agencies have, and how much security (measured in kilochildren?) should we get in return?

A huge problem with the NSA's activities is that they are, or at least have been, so secret that it is impossible to even discuss in a meaningful way the political question you posed. This should be unacceptable even to those who think catching the 300 terrorists made the surveillance ok.

no.

300 "terrorists" caught for a few billion people's privacy completely destroyed at a price tag of 100 Billion (pulled out of my ass obviously). Absurdity at its finest.

How many of these "terrorists" were just young men suffering typical teenage angst, and happened to be of the wrong religion?

"Asked how many terrorism cases were cracked using U.S. phone records, John “Chris” Inglis, NSA’s deputy director, answered that a dozen domestic terrorism investigations had made use of the records. But Inglis could cite only one in which the records were instrumental: a group of men from San Diego who sent $8,500 to Al Qaeda-linked militants in Somalia. One of the defendants in that case was discovered when a known terrorist phone number in Somalia was compared against the database, Inglis said."[1]

[1]http://www.latimes.com/news/politics/la-pn-secret-nsa-survei...

It depends. Not only does it depend on how much of a threat those 300 really were, and how much damage they would have done had this system not been in place, it also depends on how much this system gets abused, and how much the world suffers because the most influential country on the international stage that was actively pushing for greater transparency and openness has just had the moral high ground ripped right out from underneath it's feet, emboldening all the genuinely corrupt and malicious actors in Russia and China.

I don't think so but even if it is on the table "caught" and "terrorist" need to be very carefully defined.

If "caught" means convicted and "terrorist" means bomb maker or airline pilot with intent to crash a plane into skyscrapers it may be a discussion worth having. If "caught" means arrested or "terrorist" means gave $5 to an Islamic Medical Charity that turns out to have been shady it isn't even close to worth it on a practical level without even considering the principles and general human rights aspects.

My guess is somewhere in between but with people with as weak grasp of the language as the NSA seem to have I would be very careful.

Pointing to any benefits of this program is the equivalent of an ad hominem attack. Undoubtedly there are many efficient yet illegal ways to counter terrorism. No one claims this program can not be beneficial, but that the cost is too high (abandoning the rule of law, obliterating personal privacy). It's like committing suicide to foil a murder attack.

Yea...no. The problem with being a habitual liar is that people can't trust you even if you tell the truth. That 300 number sounds as much bullshit to me as all their other claims.

"We don't gather any data."

"We only gather metadata."

"Ok fine, we gather everything."

Judgement day is inevitable.

G�#��$�5�%��V��5�F�'98u�x�)�w��[_Fa��6�1f�!��['��"��VGu~w� ��|�U��Z�hep��G��^7{��K�wq��h|ڛ�m=�$L ��t� _��pM<�q��;��Y��C �M]!C�6ҝnV[�c�ϾWa�?C�M�"X*��b]0�Aļ��Li3`�P�#�j�f�u��!wb�]t_�q�&EԞw��r�.<?K��{

In the first slide, there are 25 red dots in a row on the bottom.. I wonder what those are supposed to represent?

Digital finger-printing, so that you can tell who leaked what document.

I dont want to detract from Snowden's very noble act, but I hazard a guess that Snowden knew that the documents he leaked could be traced back to him, or at a minimum a small team that he worked with.

It's been being weirdly suppressed on reddit: http://www.anonmgur.com/up/17832a6eafb09376d012090ff1b06dbe....

Every time a thread on this hits the top it gets mod-deleted.

Well then they are going to need that big data center. That is an unimaginable amount of data...

https://en.wikipedia.org/wiki/Utah_Data_Center

Yep. http://www.forbes.com/sites/kashmirhill/2013/07/24/blueprint...

interesting that is not even on CNN or MSNBC yet. Has anyone else seen coverage on US news?

Has anyone verified the accuracy of the slides? How do we know they even came from the NSA?

Let's see if NSA dares to dispute their authentity.

They came from Snowden. And the NSA is quoted in the article acknowledging the program by name.

http://s3.documentcloud.org/documents/743252/nsa-pdfs-redact...

Missing: How much did this cost? Did the government (taxpayers) overpay?

I wonder if we should try to put together a programme to try to drain the NSA of technical talent ... offering jobs or other incentives to try to persuade developers currently working for the agencies and their various contractors to resign?

General reflection:

As I recall it, our right to privacy is defined by the "reasonable expectation of privacy".

Currently, I see any such "reasonable" expectation to be almost zero.

Therefor, I have to conclude that we have already lost the right to privacy.

"Mr. <webservice-ceo> Does your company offer a backdoor for the government? It looks like it, even though you have declared that there was no backdoor just a few weeks ago!" - "Uhm. Not wittingly!"

That's like accusing AT&T of having a backdoor because your apartment has thin walls and someone could overhear. Or that your ISP has a backdoor because your WiFi isn't encrypted.

The FBI was doing this decades ago with Carnivore. Why is it at all surprising that such a program continues to collect unencrypted information you sent over the Internet?

access.

any citizen can go out and listen to public conversations, but we are not all invited to put our e-stethoscopes up to the internet backbone.

How weird is it that cnn.com is included as a reason for being interested in HTTP?

Another example is shown where they are searching for everyone who read a particular article on the BBC website.

Surely the question on everyone's mind is: how good is incognito mode???

It's not. It's only just a way to visit sites without a record being stored on your computer. Read the warning. For example, Firefox's private browsing mode states:

> While this computer won't have a record of your browsing history, your internet service provider or employer can still track the pages you visit.

They should include government spooks in that warning!

Chrome does!

>Going Incognito doesn't affect the behaviour of other people, servers or software. Be wary of:

> ...

>Surveillance by secret agents

Chromes warning does include govt iirc

The only thing incognito mode does is keep your activity and cookies while in that mode from being saved into your local browser history. It doesn't do anything at all externally.

Can someone with data center expertise extrapolate the physical scale of this operation? In terms of storage and computing power it must rival if not surpass what Google has built, no?

It is interesting on slide 17 that the NSA can decrypt all VPN traffic.

Does this indicate that they have broken HTTPS, or simply that they own VPN companies like Private Internet Access?

I don't think it's a leap to assume that ALL TLS certificates are compromised and the NSA can monitor ALL Internet traffic.

No, that's a huge leap, actually. You could go make a certificate right now for your web site and keep the private key private. I could visit your site and verify that the signature is the expected one. If you and I are both diligent, we can know that our TLS session is safe.

Wait, does the collection require FISA warrants? Is the collection still limited to foreign nationals?

It seems like working for the NSA is more like working at Inintech than it is like Minority Report. I've been more embarrassed by how Office Space retarded this seems than I have about the privacy abuses. I'm skeptical any of these supposed systems work, or even exist. It reminds me of Iran's pretend fighter jet.

Can someone explain this to me like a 5 year old and how it interfaces with PRISM?

This is considerably more invasive than PRISM.

This system logs HTTP metadata and data (think the address on the envelope and the contents of the envelope), the metadata for 30 days the contents for 3 days.

This http data is essentially everything that goes over the wire all of which is then shovelled into a database with a fairly sophisticated (if not pretty) front-end that allows really invasive searches.

You can search for stuff like "all emails that contain the words sex doll" or "nudes" and contain jpegs...of course the users would only use this system for legitimate operations covered by warrants.../s.

This is the first of these releases that have really made me stop and go "whoa" mostly because this is "better" (bigger, more complex and capable) than anything I expected them to have now (and this was in 2008).

I guess I am mostly confused about whether they have any email content capabilities...and if so, how?

They are tapped into the hubs of internet communication as well as most (if not all) of the major webmail based systems (think Google, Hotmail, Yahoo).

As to the how they are taking feeds directly off major internet routers (the vast majority of traffic will go through a major router at some point particularly if it is international though it's quite possible for a packet sent from one side of your country to another to go international as well).

So yes they do have email content capabilities (if you look at the actual slides they also have a sophisticated filtering system, they can do stuff like "show me emails from iran with word documents attached containing IAEO").

This system is absolutely terrifying, it genuinely is the work of a dystopian sci-fi author from 30 years ago.

----

If you want to get right down in the trenches email is SMTP and POP over TCP/IP (normally), email is fundamentally a human readable text protocol which makes it trivially easy to parse (this was kind of the intention after all) so once they have the captured stream reconstructing the mail is not much harder (if any) than writing a mail client.

You can see an example of SMTP if you open a console/shell and type "telnet smtp.gmail.com 25" and then when it has logged in type HELO the response is just plain text.

Wow... Govt is into big data. I wonder what they use for analyzing all this data

The funniest part is the ancient version of IE they are using in the screenshots.

What makes this even scarier is to see IE in the screenshots. :)

This bit both somewhat limits the impact and makes Greenwald et. al.'s claims that most everything is being Hoovered up a lot more credible:

"The XKeyscore system is continuously collecting so much internet data that it can be stored only for short periods of time. Content remains on the system for only three to five days, while metadata is stored for 30 days. One document explains: "At some sites, the amount of data we receive per day (20+ terabytes) can only be stored for as little as 24 hours.""

Of course, as the article goes on to detail, anything that's found to be of interest in that window can be saved permanently, and NSA analysis do that a lot.

>"To solve this problem, the NSA has created a multi-tiered system that allows analysts to store "interesting" content in other databases, such as one named Pinwale which can store material for up to five years."

It does not describe "interesting". Maybe metadata, encrypted sessions etc? Any conversations in threads linking to these articles?

But this was written in 2008. Storage capacity could/must have increased massively since then.

But so has the volume of data that NSA would have to retain.

...and so have the compression and filtering tech, I'm sure. If you strip out most attachments, the average email message is incredibly tiny when gzipped.

Notice that the presentation is from 2008 - a lot changes in five years.

One such change is coming/has come online in Utah.

That's a huge change as well as it's from all accounts a massive single purpose data centre.

Skynet right?

These slides are from 2008... I'm sure their capabilities have increased significantly.

Tempora in the UK stores all internet data for 3 days and metadata for 30 days too. Wouldn't surprise me if most western governments use XKeyscore along with data sharing agreements with each other.

I would imagine that their top priority since 2008 has been improving storage systems. It's clear that their ultimate goal is a fully indexed archive of the entire digital universe.

Yep I'm done, I'll just go about my life from now on...

It's total power, I think it's unlikely that they'll want to give up on this kind of power, they'll probably keep signing governments and 'the tech' will eventually be exported and in the hands of governments everywhere, they'll keep building this and they'll create tons of algorythms of course because it's just too much data, any resistance can be crushed... and it's so much power eventually some dark times will begin... I'm done with the topic.

Totally agree. They are not going to give this up... and worse, we are not going to fight this. We are speaking up only because this is the internet. We just want to get on with our lives.

Kinda surprising why all the people who are 'overwhelmed' and 'terrified' in the parent thread don't come out and protest. Oh wait, there's kids to feed. My bad, sorry.

(no snark)

For 99.9% percent of people, this is exactly what will happen. A few shocked moments, quickly followed by returning to all of the important things in their life. No one has time to protest all of the hundreds of wrong things we supposedly should be protesting and there aren't any clear or easy to implement steps to avoid NSA spying. There is simply no reasonable actionable item here for casual news readers.

The real consequences of this news will be seen in the actions of companies. As 'cloud' (oh I hate buzzwords) technology becomes increasingly more efficient and cheaper, as Amazon, Microsoft, Openstack and VMware duke it out over cloud customers, will those customers trust them with their data? Will companies invest in private clouds for increased security, or will large public cloud service providers be able to win over and keep their trust? How much money have public cloud service providers lost since the leaks began? How many companies are now unwilling to use cloud services from US-based companies?

There may be a protest on Sunday near you: http://1984day.com

Many hands make light work.

Note: I'm a foreigner.. :t

Credit where credit is due - NSA made useful and usable email search. Please give it to gmail and outlook.com ... I want to be able to search trough my mail as good as you guys can do.

Nice pre-emptive "attack" by Greenwald today, just before the NSA hearings.

I was worried the leaks may have peaked a bit early, but this was very well timed.

The House hearing was canceled several days ago to make time for the House Democrats to meet with Obama this morning. As far as I know they have not been rescheduled yet and I will be surprised if they happen before the August recess. Although Glenn Greenwald did say yesterday he hopes they get rescheduled in the next 24 - 48 hours.

Here is the video from the Senate Judiciary Committee hearing on the subject today though:

http://www.c-spanvideo.org/program/FISAS

So it seems.

Greenwald said they cancelled the hearing on Twitter yesterday. Do you know if they rescheduled it yet?

Just like suspected. If you use encryption like PGP, you become person of interest.

Re "Person of Interest":[0]

If yesterday we were "conspiracy theorists" when we suspected things like XKeyscore, what are we today if we suspect things like "Person of Interest"-like programs?

[0] http://www.imdb.com/title/tt1839578/

"You are being watched. The government has a secret system: a machine that spies on you every hour of every day. I know, because I built it. I designed the machine to detect acts of terror, but it sees everything. Violent crimes involving ordinary people; people like you. Crimes the government considered 'irrelevant'. They wouldn't act, so I decided I would. But I needed a partner, someone with the skills to intervene. Hunted by the authorities, we work in secret. You'll never find us, but victim or perpetrator, if your number's up... we'll find you".

(from season 1 opening)

It's scary how this is actually not fiction.

Not surprised, but where is this statement in the article / screenshots? I am not seeing it.

It's in the full presentation linked at the top of the article. Under the heading "Finding Targets", "someone who is using encryption" is one of the listed means of identifying someone for targeting (along with "someone whose language is out of place for the region they are in"…).

You've no information to back up that statement.

Using PGP as part of a filter makes perfect sense. If you're looking for "bad guys" that do certain activities, as a starting filter, it doesn't hurt to say "OK, show me everyone in this region doing these activities. Now filter by language, etc. etc.".

Just like if I was looking for gang members, I might start off a filter with "look for tattoos". It doesn't mean I'm saying everyone with a tattoo is gang member, it's just a way to start filtering.

The NSA analysts are presumably actually trying to get something done (find people they think are bad). How stupid do you think they are? If you were an NSA analyst, would you tag "person of interest" on everyone using PGP? How would that help your goal of finding actual people of interest?

They say they caught 300 "terrorists" with this program and other success stories. Presumably, they didn't achieve any success by wasting lots of time flagging random PGP users.

Read the whole presentation linked in the article.

I did exactly that, which is what I based my comment on.

Why is the commentry on this topic always braindead?

The article states that there is a query interface using the email address as the key. But Where does it say that every single email/webpage from every single person is being collected? Such a task would be technically impossible. It seems far more likely that it's querying a database of pretargeted people.

There is so much hysterical nonsense regarding this topic. The cancer of conspiracy theory spreads.

The raw slides are included in the article. You should consider looking at them.

Yeah, I've read them. Where do they say that every single email/webpage from every single person is being collected?

It's nowhere in the slides.

> Such a task would be technically impossible.

Arthur C. Clarke's first law of technology: 'When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.'

xkeyscore.com leads to Google maps.......just saying o.0

No. It shows Google Maps in a iframe and was created 13 Jul 2013. https://gist.github.com/VictorBjelkholm/6127343

Aw, Fuck.

Reply on news.ycombinator.com