Hacker News Reader

Comment by ucaetano | Hacker News Reader

The questionnaire seems pretty fair, to be honest.

If someone has access to weapons, tried to travel to a terrorist training camp, has mental instability issues, supports use of violence as well as using religion as a justification for violence and follows all the ISIS social media accounts, I'd be pretty worried.

I didn't see any questions about "does the individual holds a significant distrust of government and institutions?" or "does the individual own multiple anarchism books?". All the questions are very specific and practical.

View on HN

The guy (girl?) you describe in the second sentence is very obvious to identify. The problem is not there.

I am in Quebec City. A terrorist killed 6 people here a few weeks ago. A young, quiet, white catholic guy with clean looks. He killed 6 muslims during their prayers.

Would that questionnaire have identified that young, quiet, white catholic terrorist killer?

Perhaps not. Anders Brevik in Norway definitely checked a lot of the boxes in this questionnaire though.

Most of the questions are not religious-related, but related to violence, mobilization, etc. So if the young, quiet white catholic terrorist was doing any of those, he could be identified.

My guess is that the Columbine shooters could be flagged by the questionnaire.

But it's just a wild guess.

Does a test have to have 0% false negative rate for it to be useful? (I don't think so.)

If you apply a test like this to a few million people you will get tens of thousands of false positives. Those will clog up the works while in the meantime your quiet unassuming terrorist will walk right by under the radar and quite possibly will kill a bunch of people.

Bad tests are worse than no tests.

Your conclusion doesn't follow from your premise.

With no tests, the quiet unassuming terrorist would still walk by.

At least with a test with some efficacy at catching legitimate terrorists even with false positives would be better no?

(Going only by the metric of catching terrorists.)

You could dedicate your resources to actual detective work instead of wholesale dragnet operation in the hope that something useful turns up. Makes for good security theater though.

If your metric is simply "number of terrorists caught" then sure a broken test is better than no test. But you are ignoring the cost of "catching" innocent people. Plus you are ignoring the cost of the administering the test.

Most blood tests or tests for any major disease have a high rate of false positive, that's why you take the tests ALONG with a doctor analysing your symptoms (and that's there Bayes' theorem plays a big role).

So applying a single test to a huge general population makes little sense. But applying that to a population that already carries some traits is highly relevant.

Blood tests for diseases tend to have a much higher true positive rate as well because of substantial higher incidence rate of diseases for which simple blood tests exist than terrorism, besides that the cost of a false positive isn't that high (you just order another test to improve the odds, roughly at the same cost level as the first one).

I wasn't comparing the rates, but the process. Bayes' theorem can make predictions far more precise.

Even Bayes can't make sense of garbage input, there has to be some relevance to the input numbers.

It's not magic, it's applied statistics.

If your probabilities are on the order of 'nonexistent' and your test has a false positive rate on the order of a few 10'ths of a % then you are wasting your time no matter what the method you use to analyze your data.

Uhm, what? The alternative is to have a few million people that you're concerned about for some reason and then have no way to exclude the vast majority of them from investigative interest.

The test is useful, just not useful enough to transform a random population into a list of suspects. It's useful for what the FBI says it's getting used for - determining how to allocate resources among a limited population that's already under suspicion. Eg, be more likely to assign an active agent if the case involves people with access to weapons.

> The alternative is to have a few million people that you're concerned about for some reason and then have no way to exclude the vast majority of them from investigative interest.

If you're concerned about a few million people to spot 10 (a high estimate) per year then you will need better tools than dragnet surveillance and questionnaires like these. Did you really expect to answer any of these questions about a would-be terrorist in such a way that it would help an investigation?

Check the questions, it's positively skewed towards one particular kind of terrorist (young Islamic males even though it does not state that outright it might as well be written there). And then you get your nice, upstanding Christian white-guy who murders 6 people in cold blood and would have murdered a whole lot more if he had the chance.

These questions are meaningless without accurate priors and accurate priors require incidence rates that are distinguishable from noise.

If he truly was a completely ordinary person by all outward appearances, then no questionnaire would identify him, and this one doesn't aim to. I don't know much about the Quebec suspect, but it seems like many of the Ideology, Research And Planning, and Social sections might have applied to him. I don't think this questionnaire is biased towards one type of terrorist.

In fact it seems like the whole point of this is to help agents identify less obvious suspects, so they don't only focus on the guy who keeps traveling back and forth from Syria and retweeting ISIS.

> Would that questionnaire have identified that young, quiet, white catholic terrorist killer?

Possibly, but it's difficult to say based on your description, because age, skin color, general quietness, and religion are not on the questionnaire.

But to your broader point, could a questionnaire be written well enough to reliably identify all terrorists? If it only significantly improves detection, I'd think it would be worthwhile.

The problem with that reasoning is that if you rely on a detection method that's biased towards a specific target a competent terrorist organization can profile that method and then have a better than average change of passing through by behaving in an opposite way. That's why profiling doesn't work all that well against any sort of sophisticated enemy.

To be fair, the Quebec terrorist would have been caught on a couple of these items, but not all. Yes, this questionnaire would catch radical islamic terrorists but not the right-wing terrorist you mention, so it is only effective for a class of such possibly violent individuals.

> A young, quiet, white catholic guy with clean looks.

Has media called him a terrorist even after he has done all that ? I doubt that. He was probably identified as a person with mental health issues.

A Canadian journalist like Tarak Fatah might get flagged as terrorist though he is an kick-ass islamic reformer. Tarak Fatah has extensively travelled in hotbeds of Islamic extremism, he is associated with and has engaged all the Islamic mullahs and currently runs a TV series debate with mullas titled "Fatah Ka Fatwah".

It is hard to catch terrorists by filling up questionnaires. Some of those questions however are obvious indicators of a threat. If you already know a person has travelled to a different country to participate in violence, I am not sure why you need to fill anything else.

A thirty second Google search will tell you that yes, he was in fact labelled a terrorist by the media.

- access to weapons - has mental instability issues - supports use of violence as well as using religion as a justification for violence

The FBI is a predominantly white organization, over 88%<?> white. The rest are minorities used as pawns and analysts for infiltration and analysis of their respective ethnic populations. There's no way they're going to actively justify and develop a program to target other white terrorists/attackers/etc... They're just like the local police force except with bigger brains and some college degrees.

edit: I'd imagine the Canadian equivalent is the same since the US and Canada share a common background and approach in dealing with perceived threats to their "white supremacy".

Using encryption? Seems pretty dubious.

Also, camping and survivalism? These seem fairly orthogonal as well.

There's nothing indicating that any of those factors by itself would make you a person of interest.

And I'd guess the FBI would be a really bad service if "uses encryption" or "camping" by itself would be considered as a strong signal. The noise would be too big.

Now talk about someone who has a history of violence, takes long camping trips with a large arsenal, suddenly dropped all non-encrypted means of communication and is obsessed about books on chemical warfare and terrorist attacks. If it were your call, would you dedicate attention to that person?

The penalty for misclassifications is higher for false negatives than false positives. In fact, what happens if the FBI exhausts its budget investigating the positives from this classifier? Do they try to improve the classifier accuracy or do they seek a larger budget along with the prestige of playing a bigger role in the War on Terrorism?

More importantly, it's probably used to screen out people who don't belong. You can be damn sure anybody who is an actual terrorist uses encryption. Most harmless religious camping enthusiasts do not.

Exactly, think bloom filter. Used to rule out.

Actually, a surprisingly large amount of terrorists do not use encryption (or at least not on purpose.) For a recent example, after news media widely reported that the Paris attackers used encryption, it turned out that they actually did not [0], and instead simply used common messaging apps. That didn't stop the FBI from using the attacks to denounce encryption, of course.

In reality, using encryption isn't going to help you all that much if a government entity thinks you're suspicious. It may make it marginally harder for them to be suspicious of you, but that's about it. If they want something, and 'encryption' is in the way, 99.9% of the time they have a way around it. A very relevant, and very hilarious article on this: [1]

[0] http://bgr.com/2016/03/22/paris-attacks-iphone-encryption/

[1] https://www.usenix.org/system/files/1401_08-12_mickens.pdf

I don't think camping fits. Take out camping and you get a person with a history of violence, a large arsenal, obsessed with terrorist books. Still pretty scary.

Technically speaking I really doubt interest in camping is any higher amongst terrorists than the population.

The actual wording is:

Has the subject participated in activities that simulate military or operational environments?

So motorhome and marshmallows with your family at Big Bear Happy Camp probably wouldn't count, unless you're planning to kill everyone by making them eat s'mores until their blood sugar spikes.

Presumably none of this is taken in isolation. If all you do is use encryption and camp I doubt they're going to think you're a terrorist.

Why is camping in there at all? Do terrorists go camping more often than the population average?

Perhaps terrorists with camping skills have a higher threat potential? It's not just about P(terrorism | camping) - they also care about the expected value of terrorism plots involving particular suspects.

Like, I'm not even sure that terrorists are more likely to be known to own guns. The EV of terrorist plots involving gun ownership are higher, though, so more investigation ought to go there.

You make a great point about expected value of damage done, rather than focusing on the probability of an attack. But I think an attack with guns has a low upper limit on the number of people you can kill (maybe around 100, plus or minus?) before an armed response is mobilized. Things like nuclear, chemical, biological attacks or bombs bringing down buildings scare me more from a statistical standpoint.

I'm pretty sure that P(bombs|guns) > P(bombs), so my point stands. It's a mix of guns being dangerous, as well as gun possession being evidence of possession of other dangerous things.

Something like 30% of US households own guns. I'm not so sure gun ownership is higher amongst terrorists.

How many non-technical people do you know that use encrypted email on their personal email accounts?

There's nothing wrong with using PGP, for example, but if I found out my totally non-technical neighbor was using PGP, I would probably still give them the benefit of the doubt, but I would raise a flag in my mind and watch for other flags.

Actually, I looked at the encryption question specifically. The question is: "Has the subject engaged in or discussed tradecraft to contextually hide their online activity different from previous activity?"

So, if you're always encrypting everything, the answer here is no. Dude always encrypted everything, dude still encrypts everything.

They are flagging if you suddenly start using an encrypted communication service for specific things, as another indicator something's going on. That's fair enough imo.

No, but take a gander at 21). Does using encryption, "password-protected websites", IP Anonymization make me a terrorist? I think that describes a significant fraction of people here.

Sure, that should be taken in context to the rest of the questions, but should that even be a flag? Should we regularly detain and question pentesters now if they have a middle-eastern background?

That's hardly the only question like that. Many things on this list would, if taken in isolation, not be indicative of terrorist leanings. I'm assuming that's why there are many questions on this list.

Alright, I was about to delete my comment above when I saw a reply to another comment which had similar sentiment to mine. I guess what we need then is what weight they assign to each of these. That "using encryption" is on the same level as a listing on "express[ing] a desire to travel overseas to engage violence"[0] doesn't inspire my confidence. It makes me think that this questionnaire won't be as effective.

[0] according to the leaked document. If they weights are assigned by some other rubric in another document not leaked or part of this article at least, then I guess I am wrong.

But the document doesn't talk about "using encryption", it says: Has the subject engaged in or discussed tradecraft to hide their online activities contextually different from previous activity?

After the "check all that apply" Encryption is listed on the next line.

No, but that isn't the only question on the list. Encryption combined with fundamentalism, tactical training, and a keen interest in prior attacks is definitely a red flag in my book. And the use of encryption/privacy software definitely strengthens that case.

Fundamentalism, tactical training and a keep interest in prior attacks is a red flag on its own. I don't think that encryption adds anything.

Someone like that who suddenly starts using encryption is a major red flag. Fundamentalism, tactical training and a keep interest in prior attacks aren't enough to single out someone who's preparing an attack, but someone like that who went off the grid, started using encryption and just maxed out his credit card needs far more attention than someone spreading have from their basement.

Again, maxing credit cards and going to the grid are red flags, but I still don't see what encryption adds.

Encryption suggests active planning. Using encryption is what you do when you are actively trying to hide your activity. The others are indicators of susceptible traits - but encryption suggests imminence and active involvement. If you have gone beyond just developing radical beliefs and start actually planning an attack, that's when you're likely to start using encryption and taking operational security seriously.

Look at the actual wording:

Has the subject engaged in or discussed tradecraft to hide their online activities contextually different from previous activity?

The relevant part isn't the encryption, but the context. If you have two email accounts, one for general usage, unencrypted, and one encrypted that you rarely use, but suddenly see a spike in usage, that's a red flag.

It isn't if you use encryption, but how you use it.

Perhaps you are scared by fundamenralism. So you get tactical training, use encryption because you hate "big-brother", and anonymize your IP... Then because of your fear you research prior attacks.

I'm going to argue that's at least a tens of thousands of people if not many more. A lot of apps are implementing encryption (tens of millions of users), millions of Americans have tactical training, plus most Americans are interested in attacks.

Seems like this list doesn't really narrow down the suspect pool to meaningful numbers. More like, it makes it seem like people made this list to seem like they were doing something.

I'm not saying that this is a perfect filter that has no false positives. No one is saying that. But if you were to give me profiles of every American citizen and ask me to find the terrorists - it's exactly where i'd start.

See my reply to another comment. I concede that Encryption in tandem with the other criteria is possibly something to take into account, but it certainly should not be in the same category of things like "expressing desire to travel overseas to engage in violence." On reading the first page, it looks the questionnaire answers are weighed, but I certainly hope that "wanting violence" isn't anywhere near the same weight as "encrypts his shit."

I imagine it's not a weighting system per se. I don't know any more than you, so all I can say really is that if I were designing it, encryption would be considered a major red flag one someone who already had the other characteristics.

That is, if someone encrypts, that means nothing. But if you are comparing two fundamentalists predisposed to violence, and one of them uses crypto and privacy software and the other doesn't - I would lean heavily towards investigating the crypto guy. He's likely planning something.

I guess what i'm saying is that I think the interaction of a feature like crypto should be nonlinear with certain other features.

To be fair that describes a lot of people in the US Military.

Reply on news.ycombinator.com