Journalists Need To Start Asking About Storage, Not Access | Hacker News Reader

Journalists Need To Start Asking About Storage, Not Access

uncrunched.com

263 pointsprostoalex50 comments

I remember during the Boston bombing investigation, Tim Clemente, a former FBI counter-terrorism agent, told Erin Burnett on CNN that they could go back and get access to the content of the calls between the deceased bomber and his wife, Katherine Russell.

After some Googling, I found a partial transcript of the CNN interview...

"Almost immediately Erin Burnett, the host of CNN's Outfront, wanted to know how the government knew. Aren't phone calls supposed to be private? She interviewed Tim Clemente, a former FBI counter-terrorism agent on May 1, asking:"

  Is there any way … they [the federal investigators] can
  try to get the phone companies to give that up … It’s not
  a voice mail. It's just a conversation. There’s no way
  they can actually find out what [was said on the call],
  right, unless she tells them?

  Clemente:  There is a way. We certainly have ways in
  national security investigations to find out exactly what
  was said in that conversation. It's not necessarily
  something that the FBI is going to want to present in
  court, but it may help lead the investigation … we 
  certainly can find that out.

  Burnett: So they can actually get that? … that is  
  incredible.

  Clemente: Welcome to America. All of that stuff is being 
  captured as we speak, whether we know it or like it, or 
  not.

Source: http://www.thenewamerican.com/usnews/crime/item/15340-boston...

When I initially saw this on CNN, my first thought was, do they also have access to all photos and videos that are taken and transmitted online?

And if so, couldn't they stitch together a multi-angle montage from all the photos and videos taken at the scene of the Boston bombing (like http://photosynth.net), rather than asking everyone to manually scour through their personal footage?

UPDATE:

CNN Interview Clip (http://www.youtube.com/watch?v=pPHZrVPt4-U)

CNN Follow-Up Interview (http://www.youtube.com/watch?v=vt9kRLrmrjc)

This cannot be voted high enough. Astonishing, there one former FBI employee admits it, only weeks later everyone with a straight face denies it.

I am assuming that if something is technically possible, they're doing it. So, they're tracking telecom metadata and storing content because they can.

In fact I believe that the only reason our physical mail was mostly not read had nothing to do with the 4th Amendment, and everything to do with that it was logistically impossible and still have a working postal service, and difficult to keep secret.

With modern mail sorting machinery, I'm sure there is metadata being collected on physical mail.

>So, they're tracking telecom metadata and storing content because they can.

Correct me if I'm wrong, but to theoretically record every single phone conversation in America would require unbelievable amounts of data storage, would it not? Even if it's stored in a highly compressed format.

The metadata I can believe, but I find it kind of unlikely they have the means to store the contents of every call.

Petabytes/day, certainly.

According to this article (https://sumanrs.wordpress.com/2012/04/14/youtube-yearly-cost...), about 76 petabytes is stored to Youtube per year. (calculated from the released info, that "one hour of video is uploaded to YouTube every second."). So Petabytes/day is probably doable for them.

Whatever they're technically capable of doing, I assume they're doing it. If they can't store everything yet, I assume they're working on it. I also assume they're somewhat ahead of whatever we commonly think is possible.

Maybe not everything, maybe not now, but close to it and soon.

Others on HN have done the math on this. It's doable, at bad but reasonable quality, with 2% of the storage capacity of NSA's new datacenter corresponding to roughly one year of all American voice traffic.

"Brewster Kahle, a computer engineer who founded the Internet Archive, has vast experience storing large amounts of data. He created a spreadsheet this week estimating that the cost to store all domestic phone calls a year in cloud storage for data-mining purposes would be about $27 million per year, not counting the cost of extra security for a top-secret program and security clearances for the people involved."

http://www.schneier.com/blog/archives/2013/06/evidence_that_...

Storage is cheap, and only getting cheaper. You could archive things older than a certain date onto cheaper media. Or consider converting audio into text-transcripts after a period of time, and you can dramatically reduce your storage requirements. Either way, $27 million isn't really much.

> Correct me if I'm wrong, but to theoretically record every single phone conversation in America would require unbelievable amounts of data storage, would it not?

GSM voice codec is 9600bps = 1200 Bytes/s

x60 sec, x20 min per day, x365 days per year = about 500MB per person per year.

times however many people there are in the US, that's not a whole lot of storage and definitely a mere fraction of the NSA's data-centres.

Dan Carlin pointed this out on his podcast days after it happened. Not surprising to those paying attention, yet amazing how these guys can slip up and only a few notice.

Exactly this.

As I've commented elsewhere, words matter. They matter a lot. Especially when discussing the critical issues at play in this situation. Almost zero media outlets are asking the right questions. Instead, they all appear to be asking questions that specifically serve the talking points that were leaked a few days ago.

Perhaps media outlets and their journalists should start treating political actors the way prosecutors treat defendents--ask the right questions, vary them, alter the wording, etc., in an effort to suss out the truth and leave as little wiggle room for interpretation as one possibly can.

When shit like this hits the fan, the press must stop treating politicians with kid gloves, only levying inquiries that serve the established talking points. Go on the offensive. Ask something that hasn't already been sound-bited and parroted by everyone involved. Ask the right goddamned questions.

A few things worth asking:

1. Is the government storing, parsing, analyzing, transcribing, recording, translating, or any other type of data-gathering and/or analysis actions on the private communications of any persons who are not covered by a standing warrant. Be specific about the words chosen in the question. Even better: ask the question repeatedly, substituting each word in place of the prior one asked.

2. If so, what is the legal reasoning and constitutional authority by which the Congress has legislated such actions?

3. If so, what is the legal reasoning and constitutional authority by which the Executive is interpretating Congressional legislation to permit these programs to exist?

4. If not, what is the legal reasoning and constitutional authority that proscribes such programs from existing?

5. Carefully and completely delineate how Americans are to understand their Fourth Amendment rights in light of these programs. What does the government believe to be their papers in 2013? What level of security and privacy can Americans reasonably expect to have, against which intelligence programs will not and do not transgress?

Etc.

I think the argument goes one level further: can we allow private companies to store such data? There's no use in regulating what the government can store if they can simply seize 30 years of history from a private company's database.

This whole storyline is making me believe that data aggregation imposes a negative externality upon society. Regardless of the original intention, the threat of misuse will loom so long as the data set persists.

Excellent point worthy of further consideration and discussion for sure.

"data aggregation imposes a negative externality upon society"

This is obviously true. It is also obviously true that data aggregation imparts a (perhaps equally) significant positive externality to society.

How is that obvious?

So you're suggesting journalists to do their job? Nonsense!

Recently in Spain a comedian surfaced as a journalist, "El Follonero" (free translation: The Trouble-Maker).

It all began as a TV comedy programme about the spanish elections, where El Follonero (actually Jordi Évole, he's no longer El Follonero) asked uncomfortable questions. It eventually became a full programme, not much about comedy but with lots investigative journalism and uncomfortable interviews.

A comedian saving journalism.

Meanwhile, traditional media is just parroting the official views of the established power.

Thanks for the tip on that. I'll certainly have to look El Follonero up. Sounds interesting.

The US has this as well

Jon Stewart, Stephen Colbert, and the Onion.

Sadly, they're some of the best journalism happening in the country at present, though I'll give a hat tip to Glenn Greenwald (though he's not in the country), Democracy Now, and a very few others.

And, sadly, jester as truth teller is an ancient tradition.

Much of the political (national) and business press have long since sold out their morals and credibility for the golden ticket of "access". I include PBS and NPR on this bill of indictment.

He's actually not like Colbert or The Onion.

We also have lots of comedians doing that kind of political satire, but most of them just scratch the surface and comedy often blurs the real issues.

Jordi Évole is no longer a comedian, though he started his career as one. Now he does serious interviews and, even if comedy is still present (he's just funny and charismatic), his programme is actual journalism and not just satirical interviews and puns.

I think what gives him the edge over traditional journalism is how he approaches interviews. He looks like the average Joe and is not very knowledgeable, but he asks the questions that matter. Being very charismatic and polite, their interviewees feel confident with him, which he uses to his advantage (lots of slips from interviewees).

He also questions like a lawyer would, asking again and again until he gets an acceptable answer instead of the common PR crap that politicians are used to vomit.

The comedians I names are among the better known, and are just given as current examples. There are others, including those who spoof the news (the Yes Men for example), or who would critique it (George Carlin, Lenny Bruce).

The more general point is that humor can sweeten the bitter pill of truth.

And then there's Beppo Grillo and his Five Star movement in Italy.

This is exactly why I created Techendo (http://techendo.co -- shameless plug) . Honestly, I was fed up with journalist covering tech, who knew nothing about tech and had no technical background. We need a media source for technical news that actually understands what it's covering.

The problem is balance. Balancing both covering in-depth technical news, and also providing content worth coming back to and sharing. I'm still unsure of how to do this -- as we're still doing a ton of discovery currently.

Their talking points seem to be sticking close to these two phrases: "We do not listen to your telephone calls," and "We do not target your emails."[1] Interpreted the way the intelligence community interprets these words, listening to telephone calls and targeting your emails probably does not include:

* listening to your Skype conversations

* storing your emails (it's already legal for them to access emails older than 180 days w/o a warrant) [1]

* auto-translating your emails and generating summaries [2]

Getting more cynical/paranoid:

* using these summaries and data to flag communications based on some 'suspicion' heuristic[3]

* auto-generate and robo-sign a subpoena

* emailing it to the FISA court

Given our known capabilities in these areas, and the intelligence and diligence of those in our intelligence community, it's not unlikely that they are automating a great deal of the analysis they conduct, using software that yields results by methods we'd have called artificial intelligence a decade a go.

I would love to know if it is legal for machine learning and intelligence to generate enough suspicion for a warrant. Given what happened in the finance industry with robo-signing, is it going to far to suspect that the same process might be automating the generation and approval of individualized warrants?

[1] Yesterday in an Obama in an interview said, "What I can say unequivocally is that if you are a U.S. person, the NSA cannot listen to your telephone calls, and the NSA cannot target your emails … and have not". Then seconds later repeated: "if you’re a U.S. person, then NSA is not listening to your phone calls and it’s not targeting your emails unless it’s getting an individualized court order". Why use the same words to be vague when different phrasing would have clarified?

[2] 'FBI Documents Suggest Feds Read Emails Without a Warrant', ACLU: http://www.aclu.org/blog/national-security-technology-and-li...

[3] e.g., "John Smith and someone probably his wife had a conversation about a party with James, Linda, and Osama Bin Laden on October 1st"

Yep, you're right on it. This is exactly what I've been discussing in most of my comments over the last few days. You hear, just as in every other media-saturated issue, the same key phrases being repeated to absurdity, so as to both shape public opinion and debate and to divert attention away from asking more pointed questions and having to discuss the real reach of the programs.

Except police/prosecutors can grill subjects of investigations for hours. Annoy a politician or public official and a journalist's interview is over. An interview is a chess match: It has to be strategic, and even if the subject anticiaptes a trap you still have to lead him there. In the case of a draw (most typical outcome) the journalist loses.

Yep, this is unfortunately true. This raises the odd question in my head of how much more interesting our campy republic would be if elected officials could not schedule & control their usage of the media, but instead were required (as part of the job description) to submit to such questioning by the press.

The problem, I think, lies in the ways that these interviews typically are organized around pre-defined and submitted questions that control the conversation and allow for preparation.

>If you’re not a “U.S. person,” there are few restrictions on what the U.S. government can do to monitor you. If you are a U.S. person then…

So as someone who doesn't live in the states, this has been the biggest takeaway from all this. It doesn't matter that my country is allied and friends with the US, it doesn't matter that we're not at war, the US is not and has no intention of being my friend.

My current feelings then, is that I and everyone else who isn't a US citizen really need to get our data out of there, because regardless of where this debate is going, our "alien" data will never get any respect.

I'm a US citizen who left the US. I am pretty damn sure they're not making some sort of provision for expatriate citizens, either. How on earth could they?

I mirror some of my less-important data to a private server in the US, but it's data I can afford to lose. And everything I transfer is encrypted these days. Pretty ashamed of my country, lately.

> I and everyone else who isn't a US citizen really need to get our data out of there

Futile.

While your body may sit idle in a land subject to the queen, your thoughts travel into the independent American colonies with a click of a button. This comment, and yours above it, just did.

You might want to consider why your nation is considered a US ally. Because it sure isn't just because the US takes their word for it.

Intelligence agencies promote international stability, because they mean that at the high-level strata of world organization, its still possible to have a very informed opinion of the disposition, intentions and important issues of one's neighbours. A world where we don't know these things is less stable - you can't have trade negotiations publicly if you don't have a good idea of what can and can't be asked for, you can't "trust" anyone unless you can be reasonably sure they're not planning to stab you in the back or funding insurgencies, or if they're dealing with significant internal power struggles.

And frankly, the US doesn't know you. It also doesn't care about you. It's not a person, its a massive nation of millions, much like your own is. The fact you're allies at a high-level means precisely zilch for what your personal intentions are towards the US.

so why not give us copies of all your data. then we can trust you too.

makes sense to me.

Fundamental problem: if you give me something, then I only know it's what you were willing to give. It tells me nothing of your intentions or agenda other then what I know you'll tell me.

This is why intelligence organizations exist. Because information you acquire yourself, through your own processes, is trustworthy. Information gifted is not.

It might be worth considering what other countries besides the US take advantage of data packets that happen to travel through their country for surveillance. One would imagine that the US is not the only country who does this.

If you communicate with anyone who is a "U.S. person," then I assume that that U.S. person's communications are now "legitimate" targets.

So just as Europeans are thinking about removing data and business from the U.S, Americans might want to consider not communicating with Europeans to maintain their supposed "U.S. person" immunity from surveillance.

It doesn't matter that my country is allied and friends with the US

"No nation has friends only interests." -- Charles de Gaulle

To add more water to this article: for anyone reading books, papers or recordings dating before the very last years it is obvious that public morality is changing, rapidly and a lot.

A simple qualification of an human group (think women, Chinese, Jews, kids) was just a good joke yesterday and is now an offense. Contrary to what we may believe, our times are not more free than before on many topics, and many books (eg. Trois filles de leur mère) or movies (eg. Going Places/Les Valseuses, but even 007) could not be created today.

So keeping indefinite records is very dangerous for this reason too: you may believe your behavior is proper today but who knows what public morality will look like tomorrow?

(Just as a thought experiment: maybe watching horror movie will be considered a completely crazy thing to do and horror movie watcher will be asked to cure themselve in special detox centers. I, for one, have never understood what kind of pleasure one can get from watching an horror movie.)

(Thought experiment 2: In 20 years the animal defense movement become prominent and crazy, gets over the top, becomes the governement, and start legally killing human who showed disrespect for pets in their past life, checking the "logs". As a reaction a bunch of less crazy humans pull them out and instigate a defensive law against "over-loving animal", and then any blogger who's sent a caturday pic is suspitious forever.)

(Ok, that's imagination, but some recent event did show reality can be worse than imagination.)

edit: fixed a few words

It's worse than that. Sometime last week I noticed that people giving testimony were using the word 'collect' in a very non-traditional way. They were using it to describe the process of accessing already collected (normal English usage) data.

Doubtless, lawyers were behind this gross mutation of language. Far worse than Bill Clinton's "it depends upon what the meaning of 'is' is."

They use the word "collect" in that way because surveillance laws use the word "collect" in a very specific way, and making distinctions between "collect" and "analyze" and "intercept" is important to delineate exactly what actions people are talking about:

A few definitions: to "collect" means to gather and store; to "analyze" means that a computer or human actually does something with the records; to "intercept" means that a computer or human actually listens to or records calls.

http://theweek.com/article/index/245228/the-fbi-collects-all...

Storage. How do you define it? Apparently, any way you like, even in international agreements.

Here's an example from the European Union agreement to pass traveller information to Australia (they have separate agreements for the United States and Canada):

the airline PNR data are temporarily retained, but not stored

On the face of it, it would seem that when you mix this with probabalistically presumed intelligence agency interception, it means 'stored'. Can we please have a journalist cover something related to this at some stage, anywhere, ever? Thanks.

Full FOIA response @ http://www.asktheeu.org/en/request/information_on_pnr_agreem... (PS. Great site, recommended some time ago by Wikileaks. Use it!)

Current summary (please help expand from above) @ https://en.wikipedia.org/wiki/Passenger_name_record#Internat... ... also please help expand https://en.wikipedia.org/wiki/Passenger_name_record#Internat...

Cross reference with something from the Snowden Q+A yesterday:

US Persons do enjoy limited policy protections (and again, it's important to understand that policy protection is no protection - policy is a one-way ratchet that only loosens) and one very weak technical protection - a near-the-front-end filter at our ingestion points. The filter is constantly out of date, is set at what is euphemistically referred to as the "widest allowable aperture," and can be stripped out at any time. Even with the filter, US comms get ingested, and even more so as soon as they leave the border. Your protected communications shouldn't stop being protected communications just because of the IP they're tagged with.

This implies they hoover all data direct from fibre, but with a 'front-end filter' that restricts what is hoovered based on IP address or some other (vague, often inaccurate) marker.

source: http://www.guardian.co.uk/world/2013/jun/17/edward-snowden-n...

The existence of underseas cable taps has long been an open secret, you can find articles dating back to at least 2001 about it (http://cryptome.org/nsa-fibertap.htm).

I think Snowden's distinction between "policy protection" and "technical protection" is ultimately meaningless and naive - technical protections can be changed at any point just as a policy can be. All checks on government power are "policy protections", whether that policy is set by the Constitution or law or just regulation.

I think his point is, when policy is your only protection, and the policy is "ask secret court in secret for permission to look at data" then you really have no protection at all because there is no oversight - no way to see what is going on. At least with a technical protection (e.g declare 'they are not allowed to tap into undersea cables') you have some method of oversight, because there is some physical evidence outside of the NSA.

But isn't that declaration merely policy? I get the impression that Snowden is referring to "technical protection" as in the software/hardware systems, for instance ones that would prevent any analyst from issuing any query they like.

The technical protection he refers to is 'filter at [the] ingestion points' - in other words, its a technical protection that prevents some data from reaching the NSAs data centre in the first place. Once its in the NSA's data centre, your only protection is a (weak, changeable, unaccountable) policy one. So by technical protection he's talking about preventing the data from reaching the NSA in the first place - if you can achieve that, you're much better off.

edit: here we're talking about indiscriminately hoovered data, rather than 'asked for nicely and specifically via court order' data.

What I find confusing or misleading about his differentiation between policy and technical protections is that he values the latter over the former, but it would seem that the technical protection he cites could just as easily be ratcheted in any direction if/when a new policy dictates it.

I think Snowden's distinction between "policy protection" and "technical protection" is ultimately meaningless and naive

I think you should see this in the context of his other statements. He's not making a distinction between policy and technical protections (he just mentions them both together as equally ineffective) - he's making a distinction between policy/technical protections and law - policy can easily be changed - e.g. the filters in place are loosened periodically, whereas law (public law at least) can be policed and enforced, regardless of the particular policies in place (or in spite of them) - e.g. the filters in place are inadequate and thus exceed the NSA's mandate, therefore the program should be shut down. This is what he means by the rule of law as opposed to the rule of men in another response.

All checks on government power are not policy protections, if they have the force of law and the judicial branch is not subverted (as it has been with FISA), they have a real force which policy protections (simple internal rules of an organisation) do not have. I think the difference in concepts is useful and productive, provided you believe in the rule of law of course.

I'm more concerned that any stored data about me and you can and likely will be lost or traded or simply stolen and used for any purpose a criminal organization or corporation can invent. Data that sits somewhere can be taken at any time. Imagine all of your personal/financial data and even connections winding up in the hands of someone with a nefarious intent. You don't know it, you can't sue anyone, you can't defend yourself, you are likely ruined.

Massive surveillance, whether for marketing or "national security", definitely eases identity theft.

I'm also troubled that people are accessing, consuming, using data about me without my knowledge or consent. I would like to know if someone cares that I'm meditating with the Buddhists monks or really love collecting Beannie Babies.

President Obama on the NSA when interviewed on the Charlie Rose Show

Source:

http://www.youtube.com/watch?v=IlThTTJgKYo&t=26m40s

(Link takes you to about the 26th minute mark where the issue is mooted)

http://www.charlierose.com/

(Unwieldy video player)

Reply on news.ycombinator.com