How to make sense of intelligence leaks | Hacker News Reader

How to make sense of intelligence leaks

economist.com

This is a very useful article if you want to understand the technical language used by the UK and US intelligence communities, which is often parroted by the media reporting on topics using sources that leak intelligence from those communities. This language has been standardized within their respective communities so that all parties involved (from the President to the lowliest analyst) should be on the same page with respect to the intelligence.

According to the article there are two measures:

- a probabilistic measurement and associated language, which speaks to the assessed likelihood of an event occurring

- a confidence measurement and associated language, which speaks to the assessed quality of the source(s) of the intelligence

can you post this language here? The article is paywalled

https://news.ycombinator.com/item?id=35091213

I read a book that touched on this called "super forecasters". Apparently there was a push to reduce ambiguity in briefings. I can't remember if it was Obama or a different president (may have been as far back as Carter), but they were told something by an intelligence advisor and they asked for what is basically a confidence interval. The advisor went back and found out what they were conveying as a sure thing was basically like 50:50 chance. At some point they then decided to come up with a system to make this clearer.

Per the article:

> In 1964 Sherman Kent, a cia analyst, coined the phrase “words of estimative probability”.

So that must have been even before Carter

The Economist f'ed up completely on the Nord Stream explosion. They reported as though there was no doubt Russia did it, and told a story of Russia wanting to more permanently cut off Russian gas to Europe, which didn't make sense. They wrote off doubters as unpatriotic.

The vast majority of media did, and citizens as well. Here in Germany all mainstream media suggested Russia as the likely culprit. Which then of course made it into the canon as "Russia did it", when it was always one of the lowest-probability scenarios to begin with. Russia had no motive outside of a false-flag with extremely low probability of successfully blaming the US or Ukraine. Meanwhile, the US actually had a motive, and said they would do it. Ukraine had a motive, the ability and now some ICs are pointing the finger at them.

Still our media here suggests a Russian false flag as equally likely, and people are actually eating it up.

As a media corp. The Economist just follows certain leads and filters to lie. Sometimes obvious, sometimes not so.

So if the "intelligence community" says something without these modifiers attached, does that mean it's probably bullshit?

As an example, I looked up Colin Powell's UN speech about WMDs in Iraq. There's nothing about something being almost certain or having a high confidence. He stated it was a fact based on solid intelligence.

And then no one government ever lied again! /s

Very helpful! I had previously mis-interpreted the “intelligence community”’s assessment of “high probability / low confidence” as meaning a 92.3% likelihood, whereas it actually means 75.8%.

If only I had been in possession of this arcane knowledge back when they were telling us about babies in incubators, WMDs, the domino theory, or any number of other hilariously inaccurate and/or dishonest narratives, I would have been much better informed.

Of course, the issue is that the intelligence community often assesses things as high probability when it later turns out 100% of that assessment was based on faulty intelligence planted by an adversary (counterintelligence), or informants lied because they would be paid more for better-sounding intel, or a bunch of intelligence was just completely mis-interpreted by analysts, or the intelligence agency themselves just lied because it was politically useful and they haven’t faced real consequences even for directly lying to Congress…

The point of my comment is that “intelligence community” is a euphemism for cabals of spooks - professional backstabbing liars. They are completely unreliable narrators and their word is not to be trusted. The quality of their supposed sources and supposed analyses is irrelevant.

I’m not sure why this isn’t more accepted.

There’s a meme which sums it up pretty succinctly: Sure the CIA did shady stuff in the 40s, 50s, 60s, 70s, 80s, 90s, 00s, & 10s but they’re good now!

Because they own the mass media: https://www.cnn.com/profiles/robert-baer-profile

See also the Frank Snepp interview linked elsewhere in this thread.

I mean, based on their stated confidence we'd expect them to be wrong at least 1 in 4.

I think the main point here is that while faulty intel is always a risk, the biggest problem is willful misinterpretation. For how things played out in Vietnam, Frank Snepp's book seems to be the most well known, perhaps because it came out early. A more forceful account and criticism by a CIA field analyst is Sam Adam's War of Numbers (1995). (https://www.goodreads.com/book/show/2471957.War_Of_Numbers)

This article is missing the point. On many of these issues it isn't a question of likelihood.

Eg, the COVID lab leak theory - which will likely never be settled in the positive or negative. Doesn't matter what the percent chance is. If there is a 10% chance it was a lab leak, that is still a massive concern. We can't afford to unleash COVID every so often; there are a lot of labs and accidents happen regularly.

And even if the Havana syndrome thing is real by some weird chance; there is nothing there. It is largely a non-issue that doesn't matter. Lots of weird things happen in international politics and it isn't a top issue even assuming one of the conspiracy theories is true. It is just a cute story that people read for fun.

>> If there is a 10% chance it was a lab leak, that is still a massive concern. We can't afford to unleash COVID every so often

I try to make that point often. There were people seeking funding to do "research" that would create a virus almost exactly like Covid19. We don't need to prove that they actually did it and it leaked. They need to open their eyes and stop making bad things that can ruin the world if their precautions fail. Because sometimes they do fail.

To be fair one might need to compare the number of lives saved via lab virus research versus the number of lives lost via lab virus leaks.

Either way viruses are going to continue to jump from animals to people.

A virus research goes beyond lives saved or lost. COVID was a titanic shift in culture, economies, and foreign relations.

I’d probably take the position that lab leaks are some fraction as probable as novel animal to human transmission. However, virus mutation pressure in labs can take place significantly faster and in a preferred direction.

Labs that make medical breakthroughs through viral research reap the rewards. Shouldn’t they also bear the responsibility of making the world while when there are leaks?

https://archive.ph/dv4kV

Orwell had an interesting expression related to this, "prolefeed"

Are intelligence leaks a faction trying to get the word out? Someone trying to CYA? An attempt to manipulate public opinion by essentially speaking out of both sides of their mouths? All of the above?

When it comes to intelligence stuff conclusions are more of a probability cloud. Everyone is lying, so the question is which scenarios are more or less likely?

Instead of COVID let's take Lyme Disease as an example. Lyme Disease is named after Lyme, CT, where the stuff was first diagnosed/discovered. That's interesting. What's also interesting is that there was a biological weapons research facility across the Long Island Sound from there.

Well, if you didn't know anything else about Lyme disease it wouldn't be a stretch to believe, at some level, that the origin was some kind of bio weapon that escaped. New disease near a weapons lab? Sounds within the realm of possibility.

Now you add some more information: namely, that Lyme disease is a worldwide problem, and they've been found in ticks all over the world. Well that makes the lab origin theory substantially less likely (probably 0%), because it's unclear how ticks would have spread to Asia, Europe. Nobody is going to go around infecting ticks as a bioweapon.

So your analysis really comes down to how much information you have. You have a bunch of guys walking around randomly, not a big deal. You have a bunch of random guys meeting in random places really often, well, maybe that's something that needs to be watched.

Due to paywall, I couldn’t read the whole article. But that doesn’t stop me from summarizing it:

How you should think

The astute follower of mainstream media will sometimes notice rifts in the global narrative continuum. What was confidently touted for a year no longer seems to be true, and instead the ideas that we ridiculed endlessly seems likelier by the day. Is this an indication that we got it wrong? Of course not. Let us assure you that this is merely a temporary disruption and total consensus will be restored promptly. In particular, don’t be tempted to take this moment as an opportunity to exercise critical thinking. Remember: truths are always wrapped in complicated packaging, such as spooky “confidence levels”, which isn’t designed for layman consumption. Our most credentialed experts are working day and night to integrate these new discoveries into the narrative you all love and care for. Stay tuned.

Still, you may wonder how to make sense of such a world, with seemingly contradictory truths? Well, the only permanent cure is to not focus too much on truths and factoids, but rather subscribe to the narrative at large. It’s a lot easier than thinking for yourself, and much safer. Don’t let this wear you down though. Remember, the current truth is no more than a click away, for only $9.99/mo.

Unsurprisingly, this is not at all what the article speaks to; it may well be true that, epistemically, some of the things you're worried about are going on. But just like this is not an article about whether trace BPA in the food supply has hormone regulation effects, or whether replacing your current car with an EV is carbon positive or negative, it's also not an article about what you just said it was about.

Rather, it introduces two basic concepts in how intelligence organizations qualify their intelligence gathering: (1) estimative language: a concept that maps numerical probabilities into specific language (e.g. in the UK, "highly likely" means 80-90%); (2) analytic confidence: a concept that speaks to the verifiability of an individual source or conclusion, typically based on the mode of intelligence gathering.

Having done this, it then suggests that most individual pieces of intelligence have low analytic confidence, but that depending on the opportunity cost (loss function) of (in)action, this may be sufficient to proceed; likewise agglomerations of low confidence intelligence may form a high confidence conclusion (as in an ensemble learner).

By way of examples, it discusses the following examples: (1) whether Russia was using T-62 tanks -> whether better sights would help night vision; (2) the DOE COVID WSJ reporting; (3) whether Iraq had WMD; (4) whether Germany sabotaged Nordstream; (5) whether Russia would invade Ukraine in 2022; (6) the NIE Havana Syndrome reporting. I assume the thing that made you write the fan fiction you wrote was #2.

Honestly, it's a 13 paragraph article and a link around the paywall was posted hours before you posted your thing, so maybe just read the article?

Not the OP, but all I see is a ham-fisted attempt by some writer at The Economist to try and whitewash the "credibility" of his ideological colleagues working for the Anglo intelligence agencies (there's a very high probability that both the writer and many of the people working for those intelligence agencies have attended the same schools and hold the same ideological views).

It's a reversion to the mean of "ackchyually, they didn't really want to say that, and this is why". The percentage thing is just the scientist cherry on top.

It is likely with low confidence that the source you were most certainly replying to is referring to the front of shaping "Public Opinion" in which intelligence agencies highly likely with low confidence took part in [0] and with still many reports with low confidence most likely still are.

It is certainly interesting to learn about the Anglo-American intelligence jargon, however without detailed historical/geopolitical/technical/operational context it is nearly impossible to adequately weigh their content. The article even goes into the intricacies of communication (the assessment itself taking influence on the situation) but does not mention one of the main operational goals of publicly communicated intelligence: managing the narrative.

[0]https://m.youtube.com/watch?v=UwerBZG83YM

This is the 1983 interview with Frank Snepp about manipulating the media.

Somehow missed the link at first, just read it now. That’s a good factual summary. And yes my “fan fiction” would have been better suited against many other articles. Mostly the title “how to make sense of X” reminded me of the actual CNN headlines on the format “how to think about X”.

But, and this is somewhat speculative, I do find the timing of releasing these barrages of articles with the general sentiment of “intel is less trustworthy than you think” only now, at a time when recent data marks a stark narrative rift. Yes it’s low confidence (in the lab leak case) but that can be summarized in a sentence and then we can talk about the implications of that possibly a man made pathogen leaked and killed millions? There’s perhaps no room for that after all technicalities are covered?

OTOH, if one should assume the best possible intent, I’m sure there are a lot of people who truly missed the point and think eg the lab leak is proven true. Even so, I find it strange that media so eagerly volunteers as educators.

Even the probabilities are somewhat bullshit. 50% confidence on intelligence?

How was that number calculated? Someone definitely pulled that number out of their ass.

I actually think the words like "unlikely" better convey the reality of the assessment. Numbers imply calculation. But no such calculation is even possible given the nature of qualitative information.

We’re all screwed. The modern governance of Man vacillates between well meaning incompetence and reckless adventurism. A personification of our modern selves?

The answer to your riddle is individual men cannot keep up with the attentive demands of Power.

Without principles, integrity, and devotion how can an apparatus of men maintain amid the continuum of time?

Let us consider an objectively consistent consensus on Truth, human rights, and law.

Let us consider the incompetence, complicity, and negligence of man as a great burden and liability. Now what of his corruptibility, treachery, and ambition?

Intelligence “leaks” are only one sign that your controls are delusions of self deceiving minds.

You all really hate this comment.

I tell you from an insightful perspective, you have an integrity and comprehension of modern controls problem.

Reply on news.ycombinator.com