Why spies never discover anything useful
johnquiggin.com
johnquiggin.com
The purpose of a spy is not to "defeat the enemy", but rather to provide the leadership apparatus with a clear understanding of the capabilities and intents of the other actors in the game. (This is the theory anyway). Usually this is referred to as "reducing uncertainty". The intelligence agencies collect raw information, process it (e.g. determine its veracity, probability for accuracy, etc.) and combine it with existing knowledge to produce "product"... the intelligence output that provides the political leadership with the "best estimate" of what the other actors are capable of doing, and what they plan on doing. (capability and intent).
The author appears to have confused SIGINT, HUMINT and covert action (and also doesn't seem to have a clear idea on how spies actually operate). SIGINT is considered particularly useful because it is not subject to the whims, calculations and forgetfulness of a human agent. HUMINT is much more useful in the long run, and this is something that the Russians knew from at least the days of the Okhrana. The Russians wrote the book on HUMINT and have forgotten more about it than the West will ever know. (See "Technophilic Hubris and Espionage Styles During the Cold War" [2])
1. Russia is big, very big. Traversing Siberia would have taken a long time. Something that they would have to do without any naval support. Can you imagine trying to establish supply lines through Siberia?
2. Add continental winter on top of that and you get no go on Japanese invading USSR. (Temperatures of 15-20 degrees centigrade colder than people living near the ocean are used to).
3. Because China would obviously sit very still, when large portion of Japanese ground forces would be absent.
4. Remember no naval support? Attacking Russia would effectively split Japanese fleet and ground army.
I do not know if Japanese understood those simple facts, but Russians sure did. European front was way higher priority than Siberia. An if Japanese did attack, an order of magnitude smaller, more mobile force would had been enough to harass supply lines and stall any progress. It is not only winter time that would have been problematic, when snow melts things are worse for a while due to all the mud.
Similar story with Germans, it is not that Russians in monumental effort stopped the advance. They merely stalled German blitzkrieg (while taking heavy losses), and continental winter stopped it. I am not trying to take away from monumental effort thrown by soviets to stop the advance, but ultimately all it came down to: Germans could not advance and adequately deliver supplies and Russians could due to intact railroads.
* I am not saying those were only reasons, but people tend to underestimate importance of supply lines and role of winter.
Russian Winter is all well and good but there piles of what iffs that would have won Germans the war earlier. It was a very very close thing and knowing from inside that Japanese will not strike was of massive value
One might make the argument that Sorge's intelligence convincing Stalin was random chance, but one would be arguing against a lot of evidence that says otherwise.
There a plenty of other examples from history, although few are as world altering as Sorge. Penkovsky, for example, provided the CIA with the manuals for the Soviet nuclear missiles that were deployed to Cuba during the Cuban missile crisis. These manuals provided the JFK government with knowledge about the _capabilities_ of the Soviet missiles. In particular, they told the Americans how long it would take to prepare and launch a missile (several hours). This gave the Americans some confidence during negotiations, knowing that they would have hours of warning (from their IMINT and SIGINT assets) before the first missiles were ready to launch.
If the argument is "spying doesn't work because the odds of a leader acting on that information are the same as random chance"... that is an argument against the quality of the leadership, not the quality of the intelligence agencies.
HUMINT is not predictable because humans are not reliable. However, when you score big with HUMINT, you score very very big. HUMINT is absolutely the most devastating spying technique available.
Any discussion about "misinformation" falls under counterintelligence which involves, as they used to call it, D&D -- denial and deception. It is incredibly difficult to create a plausible deception, requiring a huge investment of resources to make it believable. There are a large number of channels that the opposition will use to evaluate information, OSINT, IMINT, HUMINT, SIGINT, etc. etc. If there are discrepancies from these sources, then the deception will fall under further scrutiny. Not really what you want to have happen when you're already unable to provide a consistent "story" across multiple sources.
Point is: HUMINT is seriously powerful stuff.
If it's cheap and easy for the enemy to produce misinformation that's indistinguishable from the real thing, then it's reasonable to assume that the enemy will do so (and it's a good idea to do so with our own information). Maybe Sorge is a good example of a type of source of information that is costly to fake. He was a trusted agent who had a long career. Maybe it would be far too costly to implant a double agent like that.
Saying spies never discover anything useful mainly displays a total ignorance for the topic.
The most famous counter-example is the breaking of Enigma, to give a SIGINT example (http://en.wikipedia.org/wiki/Cryptanalysis_of_the_Enigma).
In addition, I think the author mixes up the notions of SIGINT and HUMINT. I think we can all come up with at least one example in History where HUMINT was key to victory.
Can you come up with a few humint key to victory examples?
http://en.wikipedia.org/wiki/Operation_Bodyguard
There are many examples during the Cold War, but I'm not sure it can be said "key to victory", if there is any victory to be considered in the first place.
The crowd just loves the idea of spies being useless morons based on some game theory musings by a fellow armchair analyst.
Likewise, it's reasonable to deduce that certain kinds of intelligence gathering are useless if you start from the premise that counter intelligence is easy to generate and difficult to detect. If you disagree with that conclusion, then I think you probably disagree with the premise rather than the logical argument. I don't know anything about spying, but if you can share some insight into whether or not the premise holds water, that would be interesting. The article specifically talks about examples where counter intelligence would be hard to generate or easy to detect. What did you think of those examples? Do you have more? Do you have insight into why counter intelligence would be harder to generate or easier to detect in general?
Not the OP, but my first thought was this: http://en.wikipedia.org/wiki/Soviet_Atomic_Spy_Ring . I've now read for the first time that the program's importance seems to be "disputed", but I still think you'd have to be crazy to think that the extra information didn't help the Soviets.
Edit: Probably SIGINT, not HUMINT, but I'll leave the link for other people's reading pleasure.
In any case, one that would definitely be HUMINT- I recall some Axis pilot defected to the Allies, giving the Allies information about a new plane. I don't remember what plane- our first Zero was recovered from a crash, maybe it was a Messerschmidt?
http://en.wikipedia.org/wiki/Jessica_Lynch
I suppose you can tell yourself that they're doing this on purpose and hiding the real successes, but if so, it sounds like a daring attack on their own credibility.
Bear in mind that coming up with an example where HUMINT was key to victory doesn't actually prove your point at all, because there may be even more examples where counter-intelligence was key to victory.
Both GCHQ and NSA can point to successes as can all the major players in the spy game would Stalin have got the bomb so quickly without spies. Or the success of the XX committee. Or from the other side the Cambridge 5 where certainly effective.
And do you think binladen would not have been found and killed if it wasn't for spies run by CIA officers?
Speaking of Syria: spies and so forth were who determined whether or not Syria even _used_ chemical weapons.
http://hotair.com/archives/2013/08/26/idf-intercept-shows-sy...
It was a SIGINT Intercept. Spies are extrodinarily useful for figuring out the truth when international incidents occur. And knowing who is lying, why they're lying, what their motivations are, etc. etc. These are all important foreign-intelligence missions that Spies do their best to figure out for their country.
--------------
Now of course, spies get it wrong sometimes. The famous Iraq situation was brought about because the CIA focused on one bit of evidence that turned out to be falsified. But nonetheless, the general "intelligence successes" strongly outweigh the "intelligence failures" in the past decades.
To the contrary, he specifically contrasts humint with sigint: "
Spying may be worthwhile in cases where it is very hard or very costly to produce misleading information. Two potential cases are those of code-breaking in wartime, where the number of messages an enemy needs to send is so large that their validity can be checked fairly easily, and that of a secret weapon, where the information produced by spies can be checked by actually making the weapon."
> Both GCHQ and NSA can point to successes as can all the major players in the spy game would Stalin have got the bomb so quickly without spies.
To the contrary, he specifically addressed Stalin's bomb: "On the face of it, the reaction to the atom spies seemed justified. The atom bomb was a weapon that could destroy the world (and perhaps still will) and the science on which it was based was popularly associated with the genius of Albert Einstein. Surely, the only way the Russians could create such a weapon was to steal the secrets of the West....But once the existence of the bomb was known, any competent team of physicists, with access to the right resources, could duplicate it...The secrets passed by Western spies probably saved them a year or so in their research program but did not fundamentally change anything. The Chinese, French, Israelis and others made their bombs without significant assistance from spies."
> Or the success of the XX committee.
To the contrary, a victory for the British necessarily meant a defeat for the Nazis, a Nazi defeat only possible because they tried to use humint: "Yet the actual achievements of these shadowy regiments were unimpressive. In most cases, espionage agencies can cloak their failures in secrecy, but the defeat of the Nazis paved the way for a look at the record of one of the most-feared espionage networks in history. The Hitler regime made numerous attempts to infiltrate spies into Britain and to recruit British agents. As far as can be determined from the German records, all were captured and many were ‘turned’, being induced or forced to transmit disinformation to Berlin."
> And do you think binladen would not have been found and killed if it wasn't for spies run by CIA officers?
Sigint, again. Not that bin Laden's death really mattered aside from the documents captured and the symbolic victory.
Yes, as anyone that has ever had a security clearance knows, the biggest target of humint is sigint. The level of clearance that is needed to handle cryptographic material is higher than the clearance to know about the information sent by the cryptographic system. That's because one piece of information, no matter how sensitive, is nowhere near as damaging as being able to read all encrypted material of your enemy.
Clearances aren't just a title that you get - an increase in your security clearance goes hand in hand with your history coming under heavier scrutiny. At first you might just get a cursory check to verify that you are who you say you are - teachers from your school may be contacted, neighbours etc. As you start gaining access to cryptographic networks, the checks start going into who you associate with, what your hobbies are, what's happening in your bank account. And the checks get more discreet - you might learn about your primary school teacher getting asked questions about you, but you won't learn that your telephone conversations were listened to for 6 months to make sure you're not hanging out with $THREAT_DU_JOUR.
So if sigint is valuable, one of the best ways to get your hands on it is to compromise someone that has access - in other words humint. The two are not disassociable.
Interestingly, this is what I always say to people who proclaim that torture can't produce reliable information. The "hit him with this $5 wrench until he tells us his password" school of torture produces highly reliable information, because it's trivial to check whether what the guy said was correct.
No one ever seems to be happy to hear it. :/
e.g.: Bob hates you, and leaves a thumb drive labeled "CP" behind your computer, filled with random astronomy data. He then leaves bits and pieces that make it appear to be an encrypted drive of illegal porn. (Or, any other nefarious Jack-Bauer-worthy terrorist movie plot.)
When you get interrogated by wrench-wielding thugs, it will never be possible for them to tell whether you are lying about whether it's yours, whether you know what's on it, or whether you can decrypt it.
Torture can only possibly get useful information if the person you're torturing HAS that information. Of course that can work -- no one doubts that. Think from the perspective of the wrongly-accused person, though, and you'll see that it very quickly becomes a game of "what can I say to make them stop hurting me". This is why so many people in medieval times confessed to being witches, or part of treasonous plots: their choice was death by a relatively quick means versus death by torture.
edit: My point is, "reliable" means more than "we can verify that it's true", but also means that you can avoid false positives.
Sounds like the difference between P and NP problems in Computer Science, i.e. P (polynomial-time) problems can be solved in polynomial time, whereas NP (non-deterministic polynomial-time) problems can't be solved in polynomial time but their solutions can be checked in polynomial time.
> How do you differentiate someone who is LYING from someone who is the wrong person, or who doesn't know?
Sounds like the Halting Problem in Computer Science, i.e. checking whether an algorithm halts for all inputs in uncomputable.
It seems Computational Complexity and Computation Theory have something to say about whether torture is an effective means of extracting information.
You are wrong. For many people it is an item of faith that torture cannot produce useful information. In my opinion, these people are generally overattached to the principle that good results come from (morally) good acts, and evil acts necessarily produce bad results.
Yes, if you torture the right people you will get useful information. People who are tortured will give up the information they have at some point.
The problem is that we often torture people who have no information, and so they tell us what we think we want to hear.
1. Elimination of opposition abroad, Trotzkyi assassination and many others.
2. Discovery that Japan will not enter war. Results in transfer of Siberian divisions to Moscow that play decisive role in war
3. Plans for operation Citadel. Key point in war in the east
4. Nuclear bomb, author discounts 1-3 years.. that had huge effect on politics around the world
Invasion of USSR failure, one of the reason for lack of reaction was that NKVD had apparently established set of citeria that would tell them that Germany of planning an attack. Winter uniforms production, non-freezing oil production and some others.. since none of those were crossed, detailed and very good humint they were getting was discounted.
Also he forgets that Nazis also broke a lot of allied codes. If I remember correctly one of the allowed them to know most of the convoy routes through atlantic
Also forget that Battle of Midway was basically won because of sigint.
Generating believable crap is not that easy.
In reality I think it was a combination of Soviet Army just being really bad initially compared to Germans(remember Germans destroyed French and British without breaking a sweat just a year before) and legitimate conflicting assessments of German army no preparing for war in Russia. This lack of preparation is what cost them in December 1941, if your tank oil freezes.. well you are not going anywhere
So its easy to read into this account that Stalin knew well that he was probably going to be attacked, and rather than 'not wanting to believe it' in a wishful way, rather balanced doing nothing to provoke Germany with taking defensive steps and decided on the former course rationally.
The fact that Stalin tried to play this off to Churchill doesn't really refute anything. You don't become a legendary dictator and erect statues and portraits of yourself everywhere without being a little prideful.
Later on, they got a lot more serious about competence in their military forces, and won the war while the victorious Germans, gloating about overcoming their enemy and having captured such an astounding amount of territory, let down their guard and paid the price.
I also believe that information needs to be acted on to be relevant. This is why corporate espionage isn't very helpful. You can throw a monkey wrench into a competitor's market trials, but IP is very hard to copy. Having a copy of somoene else's source code only gets you so far - you need people who actually understand it. Having blueprints to someone's plant only gives you clues - you actually need someone who understands it enough and can overcome Not Invented Here syndrome to make value judgments.
I recall an anecdote from the 80s where an American car company was taking apart a Lexus. They concluded that their cars were superior in every way, and that all the design decisions were a result of poor Japanese design. Whoops!
The spying that organizations like the NSA against foreign countries, even allies give key insight into what the leadership there is thinking. If you think that there haven't been big wins in diplomatic and other areas from this data, I find that hard to believe.
In warfare, there are all sorts of examples that come to mind where spying saved the day. How about the utter destruction of the WW2 U-Boat campaign via interception of communications? Or Patton's "ghost army" of inflatable tanks that combined with double agents misdirected the German defense of Europe?
to be fair, espionage and intelligence gets it wrong very often. the goal of intelligence, though, is to find some truth in an uncertain situation and the presence of active deception. this is hard enough when you're talking about figuring out another government.
I think there's an interesting argument around whether or not a "traditional" government spy agency is "nimble" enough to analyze and understand modern organized crime and terrorist groups, but it's both an argument that I think intelligence professionals have been having for quite some time, and an argument the author of this blog is not interested in considering...
And can you elaborate in which way was ivy bells catch useful?
> was helped by the Poles who had stolen a machine before the outbreak of war,
which is false! the poles actually built an ENIGMA after analyzing encrypted traffic, see this book: http://www.amazon.com/dp/0743217349
Slightly off topic, but you may enjoy this on my blog: http://williamedwardscoder.tumblr.com/post/19946053957/enigm...
The USSR's first nuclear test was in 1949. The UK had access to some Manhattan Project data and didn't detonate a test bomb until 1952. France didn't test a bomb until 1960, and that was arguably the first repeat of the Manhattan Project without data and designs from the project.
It's entirely reasonable to assume that the USSR wouldn't have had the bomb until 1964 or so without it's spy network. That would have changed the cold war dramatically.
But, whilst Russia was shattered, it diverted far bigger resources to gaining some sense of parity with the US after the war than other countries.
The USSR looked to the west as a threat; the Brits and the French didn't fear their ally the US, and didn't need to get a bomb of their own with such urgency (until the USSR got the bomb, that is. Suddenly also having the bomb was strategically important.)
Spies and traitors tend to discover/reveal HUGE secrets, e.g., how to make a nuclear weapon, how to crack air defense, exact plans and order of operations for critical engagements, etc.
Terrorism doesn't even enter onto the same radar screen as those sorts of things, because all of the terrorism in all of the 20th C amounts to what, the effects of one atomic bomb? (Back of the envelope, not terribly accurate, but gives relevant context.)
I'll stipulate that the vast majority of spies never discover anything useful. I'll further stipulate that "vast majority" involves several decimal points, i.e., 99.999% or higher.
And I'll assert that it is still worthwhile having spies for what the 0.001% or 0.0001% discover - because they don't go for the low hanging fruit, they go for and get the game changers.
In case of war or copying other technology, they may have a lot of merit, as many commenters have said. But if you actually look at the world, most of the small nations do just fine without Internet snooping. If survival of a nation would require to snoop the Internet, all of the small nations (and some are really really small) would be dead.
So it's probably a case that someone wants to be a bully, so they need the counterspies to handle all the blowback.
Then there is the tale of the Lucy spy ring discovering the impending German attack at Kursk.
Finally, the Bothans who sacrificed their lives to retrieve the plans for the Death Star.
Oops, bad example! Remember, the Death Star was fully operational and the assault was a trap. Those Bothans died bringing false information.