William Binney: 80% of audio calls are recorded and stored in the US
theguardian.com
theguardian.com
I think he meant that the NSA had access to the 80% of calls that are routed through the US, not the the NSA is recording and storing literally every single one of them. I think he was misquoted or misspoke.
William Binney hasn't worked for the NSA since 2001. Were they recording all calls back then? Did someone still there leak new information to him?
It definitely seems like he meant that 80% of calls are recorded and stored. Whether that's true or not will (hopefully) come out in time.
Bahamas: https://firstlook.org/theintercept/article/2014/05/19/data-p...
Afghanistan: https://wikileaks.org/WikiLeaks-statement-on-the-mass.html
US telecommunications is geographically disperse, and my original point anyway was that the volume of calls is enormous - storing and processing that much data is hardly a trivial problem since it's both storage AND CPU intensive. From the US black ops budget you could make some reasonable estimates as to the total size and capability of the NSA to do this.
Satellite calls have been intercepted worldwide for decades now by numerous intelligence agencies.
The average call length is 1.8 minutes right now [2]. So we've got 240 mil person * 5 calls / person-day * 365 days * 1.8 min / call. So, about 788 billion minutes of calls / year at a flow rate of about 2 billion calls / day.
At MP3 compression of 128kbps, vocal data takes about 0.94 MB / minute [3]. So, close to 688 petabytes of storage data at a rate of about 1.9 petabytes / day. Seems within the realm of doable.
The problem of analysing this is in the "ridiculous parallelism" category, so they'd just be constrained by server farm capacity. Lets say they had a system with a conservative million nodes. Each day, each node would have to process ~2 GB of audio data looking for patterns. Not even challenging. If I were clever, I'd probably run a brute force audio to text on each node, then a text to symbol pattern analyser. I'd also have higher level net processes that look for patterns in calls spatially and temporally, but with far less processors.
[1] http://www.pewinternet.org/2010/09/02/cell-phones-and-americ...
[2] http://www.statista.com/statistics/185828/average-local-mobi...
[3] http://iaudiophile.net/forums/archive/index.php/t-2081.html
And did you seriously make a link to cite that 128 kb * 60 = 0.94 MB?
So without any non-COTS tech and without sacrificing the lilt of grandma's sparkly voice, we're already at a 90% reduction of your numbers.
So let's go with 190 TB/day. Say we keep 30 days in hot disk storage, and spool it off to a digital tape robot afterward. We'd need a few thousand COTS hard drives on less than a thousand servers (triple it for decent RAID).
Then a few million-dollar LTO tape libraries, and a couple of guys to schlep tape cartridges all day long.
(Of course, since we don't require all that compute power, and because Gov't doesn't build datacenters like Google does, they'd just buy a few fat IBM z/OS boxes, and a few fat EMC cabinets for low $10's of millions total, and call it a day.)
Either way, it's well within the realm of eminently achievable, I'd say. Which means it's obviously happening. :)
On the final point, agree. This is in the range where its cost is round off error in some of the large security budgets. (+ probably far more for the labor, ops, ect...)
That was one of the main reasons I worked through it, because it sounds like a horrifically large task, but its really not even that crazy in terms of data rates and storage when you break it down.
The Utah facility gives them the ability to store calls for longer, it doesn't increase their scope.
The reason they don't record 20% of calls is, from my understanding, that they have internal rules against recording certain people important to the democratic process within the US (e.g. politicians (and their families), judges, election officials, etc).
Essentially the politicians were concerned that the NSA's resources would be utilised by whichever party was in office to gain an advantage, so they put themselves (both parties) onto a "no spy" list. But you won't find the law where it says that, because it is all internal-rules created by committee behind closed doors.
I'd love to hear how you came to your conclusion that only politicians, et. al. enjoy such protections.
What exactly used to be true that is no longer true?
That's what you said that is no longer true.
"The FISA Court (FISC) today released a heavily redacted version of its July ruling approving the renewal of the bulk metadata collection on all phone calls from US phone providers under Section 215 of the Patriot Act. "
Then there is the "3 hops" rule that allows full spying of individuals if you are "3 hops" from a suspected terrorist, which is most people alive.
I'm not saying whether or not I support any of this, but I do acknowledge that the NSA was not allowed to spy on US citizens before the patriot act and everything we know indicates that they strictly followed this rule, but things have changed.
the libertarian in me is greatly annoyed they can do this.
Aren't cats notoriously good at surviving in the wild? Both feral and domestic cats (if they have to).
The NSA's own BLARNEY initiative for this stuff dates back to 1978, although I think GCHQ beat them to the punch in effective mass telephony intercept - the first one I'm aware of started in the early 1980s, although that is offline, and probably no longer archived as the media (digital tape with low-bitrate encoded audio) would probably have degraded beyond all usefulness by now (and it sounded pretty crunchy in the first place, I understand, kinda like a bad squelchy lower-side-band transmission!).
Remember, however, that they didn't have the same kind of analysis capability back then that they've got these days (but they can probably go back and analyse old stuff they still retain). As Snowden's disclosed, you're seeing the newer systems having full-take ring buffers in nearline storage, followed by offline selection for recent access and analysis from that using a huge amount of distributed processing, and in turn automatic selected archiving out of that. It's pretty much the difference between microfiche and Google in effectiveness.
http://blog.archive.org/2013/06/15/cost-to-store-all-us-phon...
I don't see a powerful enough counterforce against this insidious trend anywhere around the world. "Inspired" by the US, other countries are joining a competitive surveillance race stoked by private corporations selling everything from GSM monitoring to big data.
</rant>
However, I don't think it'll ever devolve to the point where mass arrests or anything truly draconian takes place. There isn't any need in Democracy. You need to only control 51% of the engaged voters [e.g. the people who actually vote] and at least in the US the two party system controls that quite effectively.
I'm worried about this because it will lead to people getting stepped on and crushed between the massive gears of the "State Security" apparatus.
Unfortunately, you're wrong here. The first trivial counterexample that comes to mind: You cannot have a democratic society if a government agency secretly records the phone calls of all (current and prospective) political representatives.
In the case of a politician, from time to time, one of them would say "fuck it" and use their platform to say it to the world.
The subtler approach would be to take dirt on Candidate A and give it to Candidate B so B could win the election. However, in the real world, slinging real mud is just as effective as slinging fake mud in an election. [e.g. Swift Boat Veterans for Truth had no actual evidence of their allegations and the majority of Kerry's unit said they were lying...but it still had a major effect on the election anyway] Tbh, the fake mud may be even more effective since you can create the best political narrative you can without being constrained by verifiable facts.
So I'm really not that convinced that the costs of such political meddling [clearly exposed villainy that may force politicians to take action against the Security Apparatus or get voted out of office] is not cost effective. It is far, far more likely that the Security Services would simply behave themselves and make it clear that they wanted a large budget and virtually unlimited powers. They pretty much are at that point already.
Definitely not. See https://www.google.com/search?q=poland+recording+scandal for another trivial counterexample.
(Other than that, the argument that the NSA collecting data is less dangerous because in the U.S. political debate, facts don't matter anyway, doesn't sound too convincing to me.)
"The first batch, published a week ago, included a conversation between Marek Belka, the chief of Poland's central bank, and Interior Minister Bartlomiej Sienkiewicz, in which they discussed how the central bank might use its power to help the government win reelection in 2015. Mr. Belka said that his condition for help would be "the dismissal of the finance minister" in favor of a “technical and apolitical” one. They also discussed the creation of a law that would allow the central bank to buy government debt on secondary markets, a practice known as quantitative easing.
Under Polish law, the central bank has to be independent from political interference. But Jacek Rostowski, the finance minister, did indeed lose his job in November last year and was replaced by Mateusz Szczurek, an economist with no political experience. And new laws that include a provision allowing quantitative easing are also now in the works."
Why yes, if you conspire to break the law and get recorded, this is a bad thing for you. I'm not sure why you cited one of the few cases such a recording would be perfectly acceptable.
That is actually the valid and good purpose of the media and the security apparatus. Exposing political corruption and bringing it to justice wherever possible.
Because such leaks will be used selectively, against anyone the data collecting agencies see as a worthwhile target.
And no, blanket recording of all phone calls is not the "valid and good purpose" of the security apparatus, even if it occasionally exposes political corruption.
So because they don't remove all corrupt officials, just some, its a bad thing.
Okay. I guess I don't really see that as any different as now and is completely unrelated to the quantity of recordings.
> And no, blanket recording of all phone calls is not the "valid and good purpose" of the security apparatus, even if it occasionally exposes political corruption.
That isn't what I said. I said exposing corruption was the valid and good purpose. It had nothing to do with recording all phone calls, merely some conversations in restaurants which is what you were talking about being recorded is a valid and good purpose.
You don't get to just flip flop around and re-arrange the context of the subject you brought up [recording people having private meetings in restaurants involving illegal corruption] and [recording everyone's conversations].
These are two separate things.
Yes! Absolutely! That's the worst-case. It means that the likes of Keith Alexander and James Clapper (and others who they may be in cooperation with) get to choose the government (to a degree sufficient to suit their needs).
What would stop them from building it themselves if they were elected?
Nothing, really.
I think you greatly overestimate how effective people could be in wiping their fingerprints from the internet. There is a reason people talk about once you post something on the internet its out there forever. You aren't magically going to be able to wipe out decades of fingerprints because of a regime change, regardless of whether the NSA was collecting them or not.
You can use this argument when you can wipe out all copies of a nude woman whose ex posted her to imgur or something. Until then, you are just 100% wrong because you know you can't do what you claim.
That isn't how the internet or public communication works. The Nazis were able to identify the people they sent to concentration camps because of past associations, even once it became clear what they were doing because people don't live in a vacuum.
By the same token, you aren't going to be able to nuke all your facebook posts, twitter posts, etc. that might obviously flag you as gay "magically" whenever a fascist type government comes to power. The evidence is already there and completely out of your control.
If bad people come to power you are screwed regardless of what the NSA is currently doing. At worst, this shaves a few months off the process.
The most hilarious part of this is, all of you arguing with me about this in a public forum. Do you think you can magically go back and delete all these posts if some truly evil people came to power?
Nope.
I'm amused by the downvotes and arguments because:
1) You've publicly provided information that isn't going to go away already to show which camp you'd end up in if the people arguing with me are right. This information isn't vanishing simply because its posted publicly on the internet. It isn't going to magical become invisible in the future.
2) I've said I agree with the basic premise in my parent comment and basically said it was people who were vulnerable/edge cases that are getting screwed. That is still going to the base regardless of what happens because you only need control of 51% of the electorate to remain in power. The US electorate has shown, consistently, it will believe lies.
http://www.washingtonpost.com/wp-dyn/content/article/2008/09... "For now, there appears to be little political reason to back down. A Washington Post-ABC News poll taken Sept. 5 to Sept. 7 found that 51 percent of voters think Obama would raise their taxes, even though his plan would actually cut taxes for the overwhelming majority of Americans. Obama has proposed eliminating income taxes on seniors making less than $50,000 a year, but 41 percent of those seniors say their income taxes would go up in an Obama administration."
The ability to fabricate falsehoods and get 51% of people to believe it is a far, far greater threat than the NSA. The fact none of you seem to notice this or even care worries me more than the NSA does. If I can get 51% of people to believe any lie I tell them, I can rule any democracy in the world.
Reading my previous post you can surely infer that I disagree with the mass spying but not that I am (or not) gay. This is not a coincidence: I don't hide this political view and accept to be vocal about it publically because I believe discussing it helps protecting the democracy I live in today. If I am to be accused of having that opinion later on, then so be it because that was an accepted risk.
On the other hand I don't accept that my privacy can be used against me, be it related to my sexuality or the opinions I chose to keep for myself or only share with close friends privately.
Overall I am not denying that the information can be found elsewhere or that the Nazis used past associations (I have no opinion about that) but the key point is that you should be able to dissociate between your public life and your private life.
I am well aware of the public traces I leave on forums and "social media" but I don't know to which extend my private conversations are infiltrated and I am much less careful when I write them. So I am not sure what extreme (or not so extreme) opinion I once formulated and was stored somewhere. That could be used against me by the next fascist government. There are two ways to avoid that: the first is that it is never stored in the first place, the second is that I never formulate such an opinion. I want the option that is compatible with a democracy.
> You can use this argument when you can wipe out all copies of a nude woman whose ex posted her to imgur or something. Until then, you are just 100% wrong because you know you can't do what you claim.
That is really what it boils down to. You can't erase all of the evidence on the internet, so you are screwed anyway.
I think the most amusing part of this is the fact I basically agree with the comment I responded to except to the severity and scope of the danger.
Vulnerable people will get screwed. Most people won't.
Additionally, while the people capable of building this infrastructure might refuse to do it under a fascist regime, they're comfortable doing it now under a democracy.
It almost happened once again with 9/11. Bush basically gutted Habeas Corpus, since he decided that anybody arrested under suspicions of terrorism wasn't allowed the protections afforded under it. The government could arrest you, send you to secret prison, and that would be the end of it since you were never allowed a trial.
True, none of the above examples are true fascism. But, they are scary steps in the right direction. Fear is an excellent political motivator, and with our fear-mongering media we have, that's a more powerful tool than ever. Invent an enemy scary enough, and it seems people will actually hand you their rights while thanking you for it.
Sorry, what was that?
"Nearly half of black males and almost 40 percent of white males in the U.S. are arrested by age 23..." [1]
I do understand what you meant by "mass arrests", am I'm taking you out of context... but still I find the above statistics alarming.
1. http://www.eurekalert.org/pub_releases/2014-01/uosc-sho01031...
And EFF is NSA centric - is there a UK equivalent? Why are GCHQ getting insignificant heat?
http://openrightsgroup.org/ https://privacyinternational.org/
Several comments posted before I read all those comments and read the fine article write about NSA blackmailing politicians. I don't believe NSA blackmail can or will happen in general, for reasons I have mentioned before here on HN. One of the most common kinds of comments here on Hacker News about issues like this is a comment that ASSUMES that if government leaders are under pervasive surveillance they are all afraid of blackmail. But I don't believe that, because some government leaders and some political candidates are essentially shameless. Even after they are caught (by old-fashioned journalism, or by a jilted lover or some unrelated criminal investigation) doing something unsavory, they are still willing to run for office, and SOME ARE REELECTED. United States Senator David Vitter was reelected in 2010 even after a scandal involving behavior that I would consider shameful,[1] and the antics of former DC mayor Marion Barry[2] are probably still notorious enough that they don't need further discussion here. In short, I call baloney on the idea that NSA can keep politicians on its leash simply by knowing their secrets. Some politicians have PUBLIC lives full of dirt, and still get elected and influence policy anyway.
The other reason I don't believe this HN hivemind theory of politics is that I by no means assume that everyone in politics lacks personal integrity. Some politicians, I am quite sure, could have all their secrets revealed only to have voters think "Why is that person such a straight-arrow? Why not have some fun once in a while?" The simple fact is that there is value system diversity in the United States electorate, and there is personal conduct probity variance among United States politicians, and there isn't any universal way to unduly influence politicians merely through even the most diligent efforts to discover personal secrets. If politicians think that NSA is going too far (as evidently several politicians from more than one party do think), then they will receive plenty of support from the general public to rein in the surveillance. (Obligatory disclaimer: Yes, I am a lawyer, who as a judicial clerk for my state's Supreme Court used to review case files on attorney misconduct, and, yes, some of my law school classmates are elected officials, including one member of Congress. I am absolutely certain that there are enough politicians ready to mobilize to roll back NSA surveillance programs if they really think the programs are excessive in their scope.)
[1] https://en.wikipedia.org/wiki/David_Vitter#D.C._Madam_scanda...
[2] https://en.wikipedia.org/wiki/Marion_Barry#1990_arrest_.26_d...
The parent's comment, even if disagreeable in some regards, is substantial and the discussion is better for it having been made.
The more interesting point is the inverse position of the community in regards to network and systems security, in which all measures of threat mitigation are encouraged or "required".
It doesn't matter if surveillance is used only for good (evidence suggests otherwise, though); if the surveillance organizations have all the power to operate in secrecy and distort the truth when questioned by the legislature, their "trust us" assurances are wholly untrustworthy.
The NSA/CIA/FBI/etc. couldn't even see Ukraine coming! They were on their coffee breaks, those silly spooks! They never get anything right do they? So don't worry about them collecting information on you! They couldn't even find Osama! Those slackers. Don't worry be happy!"
The stupidest assumption one can make about a powerful organization is that it's incompetent and thus benign. Crack open a history book to see how benign power really can be.
This is sex dirt and drug dirt. Money dirt is crucial to the functioning of politics, and I'd like to see a similar list of politicians who have survived evidence of kickbacks, bribery, money laundering, and misuse of campaign funds. Sleeping with someone other than your wife while being videotaped smoking crack will not put you in prison.
>I by no means assume that everyone in politics lacks personal integrity.
This is a straw man.
2) It doesn't require all politicians to be be lack personal integrity and give way to blackmail, even a few can tip a balance.
3) Selective/distorted leaking could also tip the balance in places as to who gets elected.
4) Calls could be monitored to take advantage in negotiations for example by knowing what the real bottom line is in a negotiation, knowing which parts of a coalition can be picked off and exactly what they have been offered.
The question is not could the NSA take absolute control of all politicians but how many they would need to cause real corruption of the process. And I'm not saying that they are doing it now but that they shouldn't be allowed to get to a position where they could.
Regarding this in your disclaimer: "I am absolutely certain that there are enough politicians ready to mobilize to roll back NSA surveillance programs if they really think the programs are excessive in their scope."
Firstly there is no sign that they knew anything of the scope until Snowden, secondly "why aren't they doing it?" or failing that "if what we now know isn't excessive what would be?"
Disclaimer: Not a US resident or citizen.
First of all, whether they are shameless or not doesn't really matter. If sexually explicit pictures of a politician end up on the internet, his reputation takes damage. It's beyond his control. Plenty of politicians are willing to go to great lengths to avoid this. So what does it matter if he's shameless or not?
Second of all, politicians and other powerful people might have something to hide that isn't just embarrassing but is also illegal. It's certainly not beyond the capabilities of the NSA to target powerful people that oppose the NSA while ignoring those that support them.
For example, in the news you might read about a CEO that gets convicted of insider trading. Nothing to worry about, right? Well, what if the NSA purposefully targeted him because they don't like whatever it is he's doing? Using parallel construction, it wouldn't even have links to the NSA.
https://en.wikipedia.org/wiki/Anthony_Weiner_sexting_scandal...
I had to think something similar when I read this in your parent post:
if NSA is missing major activities of other countries (the Russian intervention in Ukraine) and nonstate terrorist groups (the "Islamic State" capture of much territory in Iraq), then surely NSA doesn't have the time and resources to analyze all of the data it gathers, and maybe it is not gathering as much data as some people claim.
That is, how do we know for a fact the NSA missed this? Could it not also simply have not acted on it, for whatever reasons? E.g. I know on the surface it's in the interest of the US to have "stability" in the Middle East, but even former CIA people have been warning since the buildup to the Iraq war, that some strategies and tactics can lead to more terrorism, more instability. And then exactly that happened, because their warnings were not heeded. Am I to believe that was incompetence? I know this borders on tinfoil terrority, but I really don't mean to outright claim what my questions might imply, I am just wondering and genuinely curious/clueless, I don't read news regularly, so I have no idea what the NSA having no idea about the Ukraine and ISIS stuff is referring to.
The truth is people with uncountable power will use that power unaccountably. I think often the person who gains the power originally isn't an abuser, but rather one of their successors.
The NSA needs accountability. But better yet, is removal of unnecessary power.
It might have sounded like paranoia in the past, but without evidence to the contrary, I will side with history and assume the abuses described above are happening.
Here's the indictment. It's quite straightforward.
http://extras.mnginteractive.com/live/media/site36/2007/0307...
Here's the cliff notes:
"No later than December 4, 2000, through and including September 10, 2001, NACCHIO was aware of material, non-public information about Qwest’s business, including, but not limited to" [litany of distressing concerns about Qwest's bottom line which ultimately proved dispositive in valuing Qwest].
Note the date.
Now, look at this table of Nacchio's stock sales:
http://oi60.tinypic.com/2n1tgr6.jpg
Nacchio claims to have believed that secret national security government contracts were going to rescue Qwest from their financial problems (note the implicit concession that Qwest had problems from which its financials needed to be rescued). One tie-in between Nacchio and NSA is the notion that by refusing requests from NSA, Nacchio lost those contracts. Stipulate that this is true; it's a plausible complaint. Nacchio still took the money and ran.
The pertinent questions in the Nacchio case are P(insider trading), P(getting caught), and P(getting caught | rejected NSA).
At this point in the debate it has been observed on a number of occasions that ranking officials at the NSA are dishonest. They have made comments both sworn, and in general press releases / statements (on the record) that have later been proven false; and we have reason to believe those officials knew they were making false or misleading statements at the time. So in short, the NSA, and the intelligence community in general have destroyed their credibility.
>Several comments ... write about NSA blackmailing politicians. I don't believe it will happen in general, for reasons I have mentioned before here on HN.
I personally suspect that it already happens. We know that it has happened before (it was a favorite tactic of J Edna Hoover). I see no reason to think that it can't/isn't currently happening.
>I call baloney on the idea that NSA can keep politicians on its leash simply by knowing their secrets. Some politicians have PUBLIC lives full of dirt, and still get elected and influence policy anyway.
I think that is a naive outlook, and that naughty habits of conduct actually serve the purposes of an extortionate NSA/FBI quite well. The NSA does not need to "keep politicians on a leash". They may merely need an occasional favorable vote or motion on the floor. Knowing the internal communications of congressional staffers (and their normal voting habits), they wouldn't need to risk extra exposure to get a marginal vote over the hump. Not to mention getting people like Ruppersberger, Rogers, Feinstein, etc. appointed to the appropriate committees.
The fact that some politicians run successful campaigns despite imperfect, even scandalous personal lives isn't proof that every politician / official is indifferent about their reputation, or that they are invulnerable to blackmail.
I can't really address your last points as they are your personal opinions but I think that most Americans have very naive and uninformed notions about how governance actually works, as opposed to the textbook version of it we were all taught in school.
Do you include William Binney, 30 year veteran of the NSA, in that cohort?
I've witnessed the assumptions that senior managers sometimes make in medium to large organisations that what they call 'IT' will somehow improve things or answer vital questions when in fact it is the observation of a business process with steps in detail that is needed to find the bottlenecks.
Perhaps I'm being cynical but the 'selling' of large volume data collection (even if a lot of that data has no predictive value and remains static on the hard drives in the data centre) to the politicians might be seen as guaranteeing budget.
NSA blackmail is far fetched. Of immediate concern is NSA employees inappropriately spying on other Americans. This has already happened multiple times that we know about: http://www.bloomberg.com/news/2013-08-23/nsa-analysts-intent... Lapses at the companies which conduct bulk top secret clearance checks should only raise more alarms.
The most recent doc leak demonstrates that not only was a contractor such as Snowden capable of searching and extracting data but it was done so without the NSA having any idea. No paper trail. How did someone get certain information on you or did they? No one knows. Blackmail hardly necessary if politically charged attacks reap large rewards by crippling a citizen running for office or an attorney general investigating a corrupt corporation. Anyone could be targeted: a sitting congressman, someone running for office, a federal judge, friends, children, family. Conducted in a veil of secrecy with secret oversight guarded by a national security mandate leaves no accountability: a total abandonment of America's foundation of checks and balances.
In the longer term, this behavior is more grim than that posed by a rogue individual such as Snowden. If the existence of the NSA and its spying on Americans is fundamental to national security anyone who opposes this behavior is as big of a threat as a bin Laden. It becomes appropriate to target not only Americans but also sitting government officials who pose a threat. Congress's muted reactions could leave one to believe this is already the reality. At the least, the fear is there.
Integrity and values are shallow and attach only individually to a mortal man. If an institution or government is built without transparency and without a balance of power, eventually, at some time bad people will gain control. They may be intentionally bad, seeking nefarious and self-motivated gain, or it may be unintentional: simply an absence of knowledge in a certain area which produces disastrous decision making. The Catholic church abuse scandals presented a mixture of both. Because the institution was deeply opaque and without proper checks and balances, widespread failure occurred. Even lives devoted to continuous moral contemplations, recitations and displays of charity could make up for this.
As to whether scandals are effective, one counterexample is Eliot Spitzer.
I do not mean this as an ad hominem, but just by this statement, the rest of your arguments comes off as absurdly naive. Have you considered that perhaps the NSA and all the other "alphabet" organizations don't actually care much about accomplishing their stated goal, but are instead working tirelessly to amass power and influence through the machinations of the state? The linear way in which you assess personal and organizational motivations is simplistic to the point of uselessness.
Very few people or organizations with power are doing what they say. Instead, you must ascertain their intentions by what they do. This is politics 101. There are countless examples of this throughout history, but now is neither the time nor place to enumerate them all.
The real question to ask is, how in the name of Alan Turing's ghost did the NSA, CIA, FBI and the rest of the spooks with all their capabilities not find Osama, or Saddam's WMDs, or the Russian invasion, or...? Hint: it's not incompetence.
TL;DR: First order analysis of intention based upon a person or organization's stated intent are rarely correct. Instead, empirical modelling of behavior over time is required to ascertain true intent.
I see no reason to trust you after posts like these. You have said so little with so much text. It shows that you were a lawyer.
Or maybe foreign intelligence is simply far more difficult.
I can't imagine its easy to intercept high level government communications in Russia where they can simply order US bases companies to turn over user data.
Some of the thoughtful points brought up in discussion of my parent comment of course address important issues that I didn't discuss at length in the parent comment. I should make clear that I fully agree with everyone here that if the common people of the United States (or of any other country) don't work together zealously and with mental toughness and solidarity to defend basic civil rights, then we are all hosed. I've seen one country's transformation from stark dictatorship to democracy and rule by law first-hand,[1] and that experience makes me very eager indeed never to see the United States slide into dictatorship. Similarly, I'd be happy to see the "tyranny of the Taliban"[2] reported on by the Guardian newspaper (the same newspaper that is the source for the article we are all discussing here) fall everywhere, and never return. People who have the dictatorial impulse try to spread their control as far as they can reach, and it takes concerted action to fight for and maintain freedom, which is why I frequently recommend that my fellow Hacker News participants study the writings of the Albert Einstein Institution,[3] based on the experiences of people power movements that have overthrown dictatorships.
Simply put, I am one of many Americans who has not only participated in public protests about NSA activities in view of my local police and TV cameras,[4] and I will continue to decry and protest about interference of my freedom or my family's freedom by anyone. Along the way, I am trying to engage in what one thoughtful comment described as discussions around "threat mitigation and risk mitigation," to understand better what private actions on my own part, what governmental actions by my government, and what actions by other actors will best maximize your freedom and mine.
[1] https://news.ycombinator.com/item?id=5985720
[2] http://www.theguardian.com/world/2001/nov/18/terrorism.afgha...
That's just as speculative as absolutely every other bit of speculation that's flying around, though.
If you're in the NSA's shoes, and it turns out your budget is insufficient to analyze the mass of stuff you're recording, you don't stop recording stuff. You'd use this "analysis crisis" to justify additional budget for expanded analysis efforts, especially if it takes the form of automatic systems developed and sold by contractors located in the districts of selected legislators.
My own speculation is that yes, they're recording and keeping (for a little while, at least) absolutely everything they can get their hands on, doing massive damage to the ideals of freedom and democracy and America in general in the process, but surprise surprise, all that recording simply turns out to be much less useful than everyone had hoped.
"The police," broadly speaking, have argued for (often successfully) more and more high-tech spying capability for many decades now, but it's not obvious that such capabilities have made any big improvement in public safety. We may all fantasize about magic wands that could make our jobs easier; for some reason, American authoritarians actually believe they can buy such a thing.
As a European, with a much more turbulent and close-up sense of history (including actual dictatorships in our recent past, in several countries, or the "democratic" governments involved even in actual bomb attacks against their own citizens, usually with the support of the US too: http://en.wikipedia.org/wiki/Operation_Gladio ), I'm always amused by this kind of non-cynical naivety.
I guess it's the same way a non-criminal black person in the US, given their historical experiences, comes to understand that "justice" largely caters to some privileges and biases, whereas some protected suburban white child might have a much more naive approach.
What I'm getting at is: when it comes to those kind of operations, always assume the worse. And the so called "abuses" are seldom done by some "bad apples", they are built into the system.
>The other reason I don't believe this HN hivemind theory of politics is that I by no means assume that everyone in politics lacks personal integrity.
Another case in point. My take is that tons of people "lack personal integrity", and a certain loss of it is a pre-requisite for a succesful career in politics.
But in an a higher level, it's also that whether people in politics have "personal integrity" or not is besides the point.
The system can operate without integrity, even if most of it's players have "personal integrity".
For one, personal integrity is totally compatible with doing bad -- it only requires that you believe that the "bad" has to be done for an ultimately good cause (e.g surveillance and safety).
Plus, ethical systems (that guide "integrity") have many levels, strict conformance to the law and professional ethics is just one of them. For example, a cop that plants evidence on some guy, perhaps does not do it because he "lacks personal integrity" (e.g to pad his arrest record), but because he really believes the guy is guilty and that it's his duty as a good guy to put him in prison. That the guy could be innocent, and what he's doing is morally wrong, doesn't cross his mind, because he is convinced.
And then they break them sometimes anyway.
Same with GCHQ, really.
Scary shit...
It doesn't mean I will take your money in the future. That's just fear mongering.
Now, I don't agree with what they're doing, but I could easily see something like this happening.
The answer is: a lot
But really you need to possess the capacity to conceive of at least one example of some selfish desirable evil, before you can grasp the vast potential this potent advantage of pervasive eavesdropping offers.
The elementary starting point in all these eavesdropping shenanigans is to imagine someone cheating at a simple card game. If you're the dealer, and your opponent is betting real money on the outcome of the game, would it be fair for you to look at his hand, or worse, stack the deck in you favor before the game even starts?
But population control goes way beyond the elementary sneaking of e-mails and phone calls. Total population control would carry profound and harsh realities with it, like eugenics, human experimentation, blood sports and human spectacle, secret prisons, human trafficking, sex slaves, disappearances. Trivial things like stock market manipulation become absurd, meaningless anachronisms, as corruption is endemic to the scenario. Extrajudicial killings, summary executions, sovereign leaders can wave their fingers and have people put to death.
What's it like to be a slave? That's what the ass end of total poulation control looks like. It's like being an olympic athlete for Iraq, and then coming home to be tortured by Saddam junior when you lose.
For those that benefit from total population control, every decadence awaits.
But, perhaps your rhetorical question is something along the lines of the "human rights are inalienable, and it would be impossible for them to do special terrible things, beyond the terrible things we normally fear like chemical warfare and nuclear annihilation"
Actually, no. Under a regime of technologically enabled total population control, there would emerge a class structure so rigid and inescapable that essentially a very small number of people would be born into a lavish, but paranoid existence, and the rest of the world would suffer a permanent crushing misery, that starts with ordinary poverty and gets worse. All of it enforced by geosynchronous satellites and atomic clocks with nearly the precision of spinning neutron stars.
Keep in mind that this would be a new age of technologically enforced, drone enabled population control. A North Korean gulag is bad, and it probably doesn't get much worse than being an elderly uncle to the dear leader, and still meeting an end by kangaroo court trial and an execution where your ripped apart by dogs.
Unless of course, the dogs are robot dogs manufactured by Boston Dynamics (oh, wait I mean Google). But maybe they're just drones armed with ever more accurate hellfire missiles. And that pretty much sums up the blue sky possibilities, if we fail to preserve the old-world checks and balances that are quickly disintegrating in the face of advances in modern technology. Completely obedient robotic systems enabling a truly air tight subjugation of humantity beneath an ever watchful panopticon. And wealth disparities that look like an abyssal cliff separating hollywood visions of the future from the stone age, and sociopaths enforcing the existence of the cliff perhaps for sheer amusement, narcissism, or whatever irrational human foible (bodily fluids, mandrake) worms it's way to the surface of the human factors that control these things that now influence our lives.
How many drones can be controlled from a single data center? How many drones would we need to enforce a permanent world-wide police state, and surveillance apparatus?
Technology is a force multiplier, and it permits fewer and fewer people to achieve a great deal, and possibly much more, far beyond what we've considered "super human" in the past. But worse yet, it can mutate innocuous well-meaning intentions in perverse ways. Thermonuclear bombs and missiles have kept a lid on serious global warfare for more than 50 years, but only by virtue of two psychotic power structures promising to vaporize each other at a twitch. Is that "good"?
But let's say the worst happens, and an evil megalomaniacal cabal implements a true robot armageddon. Wouldn't such a terrible sociopathic technocratic regime just cut to the chase, and kill off every last human being that dissents? Probably yes. But only if artificial intelligence proves to be a stalwart companion, otherwise much of the general population of humanity would be permitted to live, if for no other reason than fear of lonliness and inbreeding, but undoubtedly humanity would still be subjugated at the whim of the technocracy.
Anyway, to sum up, part of the story would be a short violent transistion phase, followed by a miserable but infinitely stable, robot-reinforced tyrrany. Like a marvel team-up of Kim Jong Il and son along with Saddam, Uday and Qusay all chanting "O'doyle rules!" while eating hot dogs and drinking Big Gulps and watching robots rape political disidents to death in a stadium filled with their genetically engineered clones and sex slaves.
Dare to dream.
It is an amusing, even throwaway line, but it has a horrific message - the US administration is happy to sail so close to the line of totalitarianism that it will possibly violate the constitution a trillion times. In the UK we jus found out similar legislation is unconstitutional and we are hurriedly writing another law to get round it. _sigh_
What happens when a country that really cares about it's constitution has to rush an amendment through or face civil rights violations from everyone?