All calls in the Netherlands are stored, indexed and searched for keywords | Hacker News Reader

All calls in the Netherlands are stored, indexed and searched for keywords

translate.google.com

164 pointsFather53 comments

Here's a year-old story by a more reliable Dutch newspaper, claiming 1 in 1000 phones is being tapped in the Netherlands.

http://translate.google.com/translate?hl=en&sl=nl&tl=en&u=ht...

I call bullshit. Phone metadata is saved since forever yes, but stored at ISPs, not at government organisations. There are strict regulations regarding the privacy of voice data over the phone (VoIP does not count as such though), and I don't think the secret service and military secret service (AIVD and MIVD) can do anything they like. They have more permissions, such as demanding passwords for encrypted files as long as it's not for your own conviction (while normally you have the right to remain silent), but it probably doesn't go that far. Keyword searches are probably not true.

It is however worth mentioning that we have this CIOT system which is a publicly known and automated system that actually provides automated access to name and address details of any given Dutch IP address. The system is updated with ISPs' data every morning and can be queried at will. ISPs, even the most privacy-aware one (XS4ALL) do not give statistics of how often their part of the database was queried (I asked them), but it has been made public that the database had a total of 2.6 million queries over 2010 and 2.9 in 2009. That's one in six citizens' data queried for no apparent reason.

Tech details: The CIOT system is a centralized search dispatcher, that queries systems provided by individual ISPs. A government official can enter an IP there and within seconds all ISPs have been queried and one probably returns a match.

I agree this is a bullshit story. They only store conversations when they have an eavesdrop approval.

A lot of people underestimate the amount of storage it would take to store all voice data.

True actually. Ironically, a call itself is much more expensive than storing it for 20 years would be.

They do have a lot of eavesdrop approvals though, or so I heard from a colleague. (But that still doesn't mean they capture all the calls.)

Storing voice (audio) data is not what the article says. I'd imagine you transcribe the audio to text and search in that. Storing text is incredibly easy. Besides you can throw away 99.9% of the data almost immediately.

I'm actually curious how much text data this would be per day; number of call minutes * average number of words per minute. I'd be surprised if that wouldn't fit in a reasonable cluster.

I'm sorry, but do we really think that machine transcription of millions of cell phone conversations is worth anything? How can anyone believe that after using google voice?

So you use a hybrid approach. The text transcription can be fed into programs that look for specific phrases, build up social networks, etc. And then anyone you decide you actually want to monitor you keep audio as well as the machine transcription.

The machine transcription remains incredibly valuable for broad surveillance even though it is highly imperfect.

You underestimate the CPU power needed to do this. The Netherlands has a population of 16 million, by comparison Google voice has about 1.4 million users. This is an order of magnitude difference. On top of this they only transcribe voicemail not all calls. What is the ratio of calls to voicemail?

Transcribing all voice calls to text in the Netherlands computationally could easily be two orders of magnitude more difficult than Google voice.

Commercial speech compression algorithms are hamstrung by the need to only add milliseconds of delay: they can only compress over a 'window' of tens of milliseconds. You can almost certainly do a much better job of compressing speech in batches of an minutes or tens of minutes: there is much more redundancy to remove. So if the spooks wanted to store massive amounts of speech data, they may have invested in such algorithms.

So let's estimate:

http://www.telegeography.com/press/press-releases/2012/01/09... says there were 438 billion international (because that's all the NSA collects, right?) calling minutes in 2011 (in the world... not just the Netherlands).

Aberdeen will sell you 1 PB of storage for $495k: http://www.aberdeeninc.com/abcatg/petarack.htm

A narrowband speech codec will encode calls in excellent quality (for the PSTN) at 12 kbps.

So that's 438 * 10^9 minutes * 60 seconds/minute * 12000 bits/second / (8 bits/byte * 10^15 bytes/petabyte) (using lying harddrive manufacturer's definitions of a petabyte) = 39.42 PB.

Or less than $20mln/year. Which of course is the quoted budget of PRISM.

Your not counting bandwidth, cpu, facility and personnel charges required to pull this off, raw storage is a minor part of the cost.

I'm not actually trying to imply that this is what PRISM does (no one has made that claim). I'm just saying that on a government scale, the cost of storing all voice calls ever made forever is not even very expensive.

So let's add bandwidth: the most expensive estimate I've seen is $0.019/GB <http://blogs.howstuffworks.com/2011/04/07/what-does-a-gigaby.... Let's assume the original audio is captured using G.711 (64 kbps). So that's 438 * 10^9 minutes * 60 seconds/minute * 64000 bits/second / (8 bits/byte * 1024^3 bytes/GB) * $0.019/GB = $3.72mln.

Let's add CPU: A medium-sized, high-CPU AWS instance is $0.0024/minute <http://aws.amazon.com/ec2/pricing/>. A moderate laptop-class processor can encode and decode 150 channels/core in real time <http://www.ietf.org/mail-archive/web/rtcweb/current/msg05236.... So that's 438 * 10^9 call minutes * $0.0024/CPU minute / 150 call minutes/CPU minute = $7.01mln.

Facility: The NSA's Utah facility is projected to cost $1.5...$2bln <http://en.wikipedia.org/wiki/Utah_Data_Center> and will contain a 100,000 square foot data center <http://nsa.gov1.info/utah-data-center/>. A 42U rack is about 7 square feet. Let's assume a floor occupancy of 25%. That's $2bln/facility / 10000 ft^2/facility * (7/0.25) ft^2/PB * 40 PB = $22.4mln.

I don't have a good estimate of the personnel involved, but I doubt it'd require anything out of the ballpark of the other numbers here. You could have every rack maintained and operated by its own PhD-level researcher at less than $10mln/year including all overhead and benefits.

A single JSF F-35A has a $207.6mln procurement cost (excluding R&D costs, maintenance costs, and operating costs) <http://en.wikipedia.org/wiki/F-35_Lightning_II#Program_cost_....

I think it's dangerous to go on record beforehand claiming something is "bullshit". If I've learned anything over the last view days it's that reports like these should be taken seriously and no stone should be left unturned to find out the truth. We can't just assume intelligence operations can't; we need to know they can't. Let the House of Representatives proof it's nonsense. Also, thanks for adding the bit about CIOT.

The CIOT system can not legally be queried "at will", they have to have permission. Although the bar for this permission is incredibly low, I believe there have been some cases where it was denied.

Of course some government services have blanket permission. Example 1: the (dutch) IRS.

more information on the CIOT

http://ripe58.ripe.net/content/presentations/ciot.pdf

it says 250k queries per month... kinda hard to get warrants for all of them i guess

Like I care how hard it is for them to get warrants. My question to them is why do you even need all this data in the first place?

I think that is the question we all would like an answer to, but we don't all draw your conclusion from it (that if they don't need it they won't collect it).

We have a public transport system here that functioned just fine using anonymous cards, and it got replaced by one that allows near perfect tracking of every individual using public transport. Why anybody would want to is a good question, but it is the system we've got and the data is being kept.

>I call bullshit. Phone metadata is saved since forever yes, but stored at ISPs, not at government organisations. There are strict regulations regarding the privacy of voice data over the phone (VoIP does not count as such though), and I don't think the secret service and military secret service (AIVD and MIVD) can do anything they like. They have more permissions, such as demanding passwords for encrypted files as long as it's not for your own conviction (while normally you have the right to remain silent), but it probably doesn't go that far. Keyword searches are probably not true.

Yes, because secret services have been known to strictly follow the law, and not do anything without telling you first.

Also all telecom companies must provide to CIOT all new and updated registered customer buying MSISDN's. If it is postpaid, all details including address, full name and D.O.B is given, prepaid also, but that can depend on how the customer has purchased the SIM.

Anyhow, why does it matter that much. If you have something to hide, then I'd be sweating. If not, who really gives a sh*te if people are tapping into our digital lives.

Facebook, Google and rest are just as bad as the governments. They are invading us with advertisements in all parts of our digital life.

If people are worried about it, turn your crap off.

If you have something to hide, then I'd be sweating. If not, who really gives a shte if people are tapping into our digital lives.*

Because your "something to hide" may be something that is currently legal / acceptable / non-embarrassing / etc., but becomes illegal / unacceptable / embarrassing / threatening to those in power / etc. in the future. And because governments have been known to collect data, nominally for legal reasons, and use it for political purposes, to threaten, harass and intimidate people based on their political affiliation.

All of that said, it really comes down to the principle of the thing. I've said - and will continue to say - plenty of things that could endanger me in some hypothetical future. I actually tend to be very public with most of my thoughts, rants, ramblings, and what-not, as I have an attitude of "If you don't like what I say, fuck you" directed at the government and pretty much everybody else. I have almost nothing to hide. BUT... not everybody has that attitude, and some people care more about keeping their "stuff" private. And even I want the option of keeping certain things private when the need arises. Just because I'm, say, 99% transparent (whatever that means) doesn't lessen the importance of that "1% secret". And that's the rub... everybody probably has at least "1%" of things that they do want to keep private/secret, now or in the future. And they should have the option to do that if they want.

"De Telegraaf" may not be a reliable source, but even members of parliament are asking questions. Apparently the Dutch government is preparing a massive internet interception program. See: https://www.bof.nl/2013/06/10/nederlandse-overheid-broedt-op.... Sorry, but Google Translate does not translate HTTPS urls.

The newspaper that published this, 'Telegraaf', is notorious for publishing bullshit. The article is very short, the journalist wouldn't be able to check if it's true, and the newspaper hungry to publish anything that attracts readers. Offline version of link-bait.

Yes, I got the same notion

If you are interested in this story and comments, then you will most likely be interested in the following comment and associated dialogue as well:

https://news.ycombinator.com/item?id=5830994

I fail to understand why tiny tranquil european countries without serious dangers of terrorism or organized crime even do this. They have nothing to reveal by intrecepting communications, why spend money and public credit on this?

Not even America is so high-risk if you ask me.

Are you serious? Dutch jihadists are fighting in Syria, the Detroit underwear bomber boarded his plane at Amsterdam Schiphol and organized crime is a serious problem in the Netherlands (see http://www.nokturnis.net/nokwiki/tiki-index.php?page=Organiz... )

And a homegrown Islamist network assassinated a well-known dutch film director after he made a movie criticizing the treatment of women in Islam:

http://en.wikipedia.org/wiki/Theo_van_Gogh_(film_director)

What? So you can have attacks on your country without requiring the world to go into lockdown, without having to invade several countries and without having to molest every traveler?

I agree that is irrational behavior. But saying the Netherlands is tranquil in this aspect is wrong, was my point.

My post was petty and sarcastic - however the Dutch don't seem to have the penchant for blasting the world with their dramas, but this may be a language barrier.. I did find the Netherlands to be tranquil - however I've only spent a week there, and didn't move around much. I didn't notice the things that have been discussed here over the last few day, and most are very disturbing.

The dutch took part in both operation enduring freedom and iraqi freedom. So you could say they did invade several countries.

In fact the Dutch owned New York once (then called New Amsterdam ;-) until they effectively traded it with the British for Suriname and some spare change.

And that whole Dutch East Indies and Indonesia thing still has some repercussions till this day.

The Dutch still have parts of The Netherlands overseas, including people that can vote in the Dutch elections, along with territories that are overseen not by the Dutch government but by the King (used to be the Queen).

> without having to molest every traveler

Schiphol is organised better than most comparable airports, and it has improved dramatically in the last 5 years, but the level of actual harassment can still be quite high (seems to vary by day).

It's probably the most harassing airport in Europe. I haven't been to all airports but I've been to many (>50) and Shiphol was the worst. Still better than the US though, if you're an EU citizen at least.

Maybe my POV is skewed by having MAN as home airport - security here is horrendous, slow, invasive and obsessed with trying every new braindead security scheme under the sun, including the infamous pornscanners.

Schiphol was not that bad the few times I've used it (last three weeks ago), IIRC they do the actual scan at gates which I think is always the superior option.

Completely agree on the EU/US comparison, of course. You can actually see how European aviation has now fully caught up and often surpassed its US counterpart -- US infrastructure looks old and creaky, often not fit for purpose, and thanks to TSA it now feels more like a series of gulags than a 3rd-millennium transportation network.

I've been to Shithole 4 times (I try to avoid it though). I've been subjected to the pornscanners every time.

You do know you can opt-out right? They actually have forms and posters hanging up informing you of your right to opt-out.

All outbound travel to the US has to go through the porno scanners, each and every time I have flown to the US I have opted out, just like I have opted out each time I get asked to go through the porno scanner in the US...

They don't make a fuss about it either, they just ask you to stand with your legs spread wide and arms spread and do a pat down/magnetic wand.

Not as bad as the US where you get put into a sort of metal "hot box" where you are told to wait till they can find someone with a low enough IQ to grope you, followed by the question if you really want to opt-out, and the agent doing everything in his power to make it as uncomfortable for the person they are checking as possible. Only that tends to backfire on them, cause most of those idiots are giant homophobes... so playing with them while getting groped is kinda fun.

"without requiring the world to go into lockdown, without having to invade several countries and without having to molest every traveler?"

Have you ever been here? There are cameras on all highways that recognize license plates and track the movement over every car on the road network (cfr: the ease with which they tracked the whereabouts of the father who killed his children a few weeks ago), it has the most phone taps per inhabitant (to the extent that if you look at the data, it seems like somebody accidentally types a few numbers too much when typing it into Excel, that's how far we are removed from the runner up), and in general the amount of information the government has on every citizen is staggering (e.g. the age at which you first grew pubes (!) ).

At Schiphol you are searched at will - a few years ago I was on a flight back from the Caribbean and everybody on the plane was searched - thoroughly, more intense than when you are searched e.g. at sports events. Every city has autonomous authority to designate certain areas as 'search at will' areas - police can (and will), without cause, search you and your belongings (including your car - US doctrine about the car being an extension of a man's house? Hahaha, yeah, in the sense that it doesn't take much to search your house, either...). Most train stations are such zones, but most of the city centers of the bigger cities are, too (let that sink in - there a whole city center where police, without any cause, can search you and your belongings!)

Oh, not carrying ID (anywhere)? €90 fine, and you can be taken into custody until you have proven your identity. No cause necessary for asking for it, either.

Invading has never been a strategy generally employed here, the Dutch are merchants, war is bad for business (I'm sure there's a fitting Ferengi quote here...) Plus a country the size of a flyspeck on a global map just doesn't generally have the muscle.

Look, I still love my country, and for all its flaws it's still the best place on earth to live for me - but let's not kid ourselves, the surveillance state is alive and kicking here, and the 'OMG the terrorists are coming' sentiment is, too.

Fitting Ferengi Quotes:

Rule of Acquisition 34: War is good for business. Rule of Acquisition 35: Peace is good for business.

Yes I have visited - only for a week or so and I didn't travel around much. I flew in from London, and didn't notice the cameras etc which I did in the UK. I thought they were bad there. My sarcastic post has had a few very good replies. The disturbing behaviour of the state mentioned by you and others is oddly fascinating. Why do it? On a pure cost versus benefit analysis, how can it possibly be justified? Those that know The Netherlands have radically changed my views over the last week with facts and anecdotes about this. Please keep posting, I for one have been educated.

I should put a nuance on my remark that police can ask for ID without cause; after looking a bit deeper into it, this is not entirely true. There has to be cause but that is defined very broadly. Examples mentioned on the government website on when police can ask for ID, and when they can take you into custody when you can't, include 'a car driving around at night in an industrial area', 'there is a shooting in a pub and for the inquiry it is necessary to establish who are the witnesses', 'youth are being a nuisance in the neighbourhood', 'there is a fire and the arsonist might be amongst the spectators' (!).

Not quite 'papieren, bitte' yet, and not as dramatic as I made it sound, but still not what one would expect in a land claiming to value freedom. (my other points still stand though, including those about the 'no cause search' zones)

I think it goes a bit far to call a couple of nutjobs 'jihadists'. And the fact that someone traveled through one of our airports doesn't mean we have a structural problem.

Of course they are nutjobs and you could probably call them other names as well, but my point is that they went to Syria to do 'jihad', so they are jihadists.

http://en.wikipedia.org/wiki/Hofstad_Network

Exactly because it's tiny

Going off on a tangent, perhaps the Netherlands doesn't have serious terrorism or organized crime /because/ of these measures, quelling them before they even go public?

Well, that's the million dollar/euro question isn't it? It's a bit hard to verify with all the secrecy going on. And arguably without some level of secrecy you cannot have effective intelligence. It's a bit of a catch-22.

ECHELON is a global communications interception system, created by the United States, the United Kingdom, Canada, Australia and New Zealand to routinely and indiscriminately monitor and record all forms of electronic communications worldwide both military and civilian and overseen by the National Security Agency. Designed during the cold-war, ECHELON primarily intercepts worldwide non-military communications, including those from governments, organizations, businesses and individuals. It could intercept practically any communication between countries anywhere in the world. The project ECHELON receiving system thieves this streams of millions of communications every hours to massive rez of computers. These computers decrypt messages when necessary, than when required utilize optical character recognition or advanced voice recognition techniques to extract words from each message. Every message captured is analysed for keywords or phrases found in the ECHELON dictionary. Keywords include all the names, places, code words or subjects that might be of interest. There are second search lists for each member country. Messages acquired at any of the receiving posts, containing requested keywords are automatically past on to intelligence organizations requesting those keywords. Those messages are flagged for further analysis. ...and ray of receiving stations collect all international communications carried by approximately 20 INTELSAT satellites. The INTELSATs are used by telephone companies of most countries. Thou they carry primarily civilian traffic, they also carry diplomatic and governmental communications. These INTELSATs are positioned in the stationary orbit around the equator and carry tens of thousands of simultaneous phone-calls, faxes and e-mails.

The Telegraaf is not the most reliable source.. just saying.

http://youtu.be/vyUQ0Z5hyU0

Reply on news.ycombinator.com