How NSA and GCHQ are tapping internet cables
electrospaces.blogspot.com
electrospaces.blogspot.com
"MI5 abandons multi-million pound IT project" http://www.telegraph.co.uk/technology/10053240/MI5-abandons-...
NSA etc are military operations with honeycombed classification schemes and cellularized access to information. Even if there were failures and waste, we'd never hear of it. I assume that corruption plays less of a role since the concern is on effectiveness, but I have nothing to base that on.
The gigantic failures exist so that when there is an actual need for aerospace engineers to respond to an actual threat, rather than the theoretical projections based on the data from the last war, it won't take 25 years to train them up from scratch.
The failures are not quite as horrible when you realize that it could be more expensive in the long run to do nothing at all. But then the questions turn toward the amount of pre-emptable core capabilities we need to maintain. For a long time, the doctrine in the U.S. has been the ability to sustain simultaneous nonnuclear war in two separate theaters plus nuclear second-strike capability sufficient to create a MAD deterrent. That is expensive. And since we haven't actually needed that much war materiel since 1945, that leaves a lot of room for waste.
It could be argued that the limited regional wars of choice that the U.S. has pursued since then have simply been capital maintenance, to prevent the military from decaying into expensive uselessness. I think in its deepest backrooms, the generals still gauge their effectiveness by whether they could fight both Russia and China at the same time.
I understood it to be a bit more specific than this: sustain conventional war against our two strongest non-allies.
In any case, it's also my understanding that we've been maintaining a level far higher than what would be required for this.
Look at the Iran-Iraq war for the level of casualties that some potential enemies are capable of taking (half a million fighting age males dead on Iran's side). North Korea having 9 million military age personnel brought up in a personality cult. Let along China and the size of their armed forces.
The USA lost four thousand in Iraq, two and a half thousand in Afghanistan. The UK lost nineteen thousand men dead on the first day of the Somme in 1916, do you really imagine a Western nation losing that many for any conflict today? To be honest even in a war of national survival I doubt it.
We lack the political will to sustain conventional war. The military knows it and acts accordingly.
Your reply seems focused on the quantity of soldiers. This is an important factor, but I was primarily considering overall spending, which obviously includes lots of capital expense, logistical stuff, etc., aside from the personnel. I believe that the level of investment in military hardware and the like is significantly higher than would be required for a successful war on two fronts (assuming that there exists such a thing as "successful war" these days).
It's why the Iranians had over 1 million casualties taking on Iraq. And several years later, the US rolled over the Iraqi military with 1,000 causalities.
So sure, the US can't successfully invade China, but it would sink the entire Peoples Navy and defend Taiwan, Japan and S. Korea.
We don't do that anymore... Because a full total war is too expensive -- when the US suffers a loss of 20,000 men in a day, a button will be pushed in a silo somewhere and cities will vaporize.
That's why North Korea exists today.
There's absolutely no reason to believe the spy tech isn't as poorly implemented and terrible value for money as the examples you mention. Given their secretive nature it's even more likely to be the case I'd suspect.
There's a lot of federal tax money spent on what is cargo cult science and cargo cult intelligence. The NSA has the ability to buy a lot of PhD's but they are stuck with the same batch of incompetent and profiteer federal contractors the rest of the government is stuck with.
These days too many of them are a bit intimidated by the mere presence and aura of people in uniform, intel people, cops etc...Years ago, many of them had served in the military or a had a very good understanding of what they can really do (not what they claim they can do) how they worked/the types of people behind them(JFK, Churchill etc) so were rightly sceptical of their claims. I mean, has anyone ever met a cop/spook/politician who didn't want more power to do X, more money to do Y, more people, more secrecy etc?
One of the main problems for intel agencies is that they are merely one conduit to policy-makers, amongst many these days. Ultimately it is those policy-makers who need to (and increasingly are incapable of) using this information for strategic decision-making...Tactically it is easy to say "arrest/shoot that guy," because they get kudos and less blame if some thing goes wrong. Strategically it is harder to say "don't shoot that guy, keep him alive and in 5 years we can negotiate with him." It took the British a long time to learn how to do this in Northern Ireland, and it was only when they did that they managed to start to move towards peace with the IRA. I'm not sure would that be possibly in the bloggy, politically polarised, short-term, tabloid, Sky/Fox News type of world we live in these days.
After all, even if they did predict the rise of ISIS, Boston, it is the policy-maker who must decide what action to take. Difficult when:
a) There are sooo many warnings coming across their desk
b) Limited resources
C) Not many real policy instruments available. For example, the US public would not have been up for attacking for acting on intel available about ISIS 18 months ago, thus that policy option was thus not available.
These capabilities are every bit about decision advantages and sabotage on international and geopolitical levels.
* Brazilian competitor PETROBRAS hacked by NSA on behalf of US oil companies.
* German elite, including Merkle, spied on during the Eurozone crisis.
* Iranian nuclear efforts sabotaged for several years by Stuxnet, and then by other cyber weapons.
* Syrian Air Force hacked and grounded during ISIS and civil war activity.
* "FISA" & FISA courts = Foreign Intelligence Surveillance Act
It makes me wonder exactly how much worse it is in the Intelligence industry. I agree with others in this thread, given the secrecy involved I expect it to be orders of magnitude worse than what we see in the private sector.
It also makes me wonder about the private Intelligence industry, about how much better they must be than their government counterparts, and about how they work together with government to "fight terror" and other demons. Combine that with the idea that government IT is losing talent to private firms in the same way... Maybe we should be more worried about private Intelligence firms than we currently are.
This is true of the criminals caught via FBI counterterrorism's more traditional police work that ends in civilian courts.
It remains entirely possible that the intelligence establishment enjoys some success in penetrating more sophisticated groups, but doesn't write press releases about it (since it will be acted upon with further covert operations to get further into those organizations, or with secret assassinations.)
Stop apologizing for the government, whether conscious or not.
Because it's being read by programmers, a significant portion of whom are "libertarians." Just ignore them.
So that generalization doesn't apply so cleanly here anymore.
That seems like "greed-heads, nerds, and old-school Usenet trolls."
Government activities are generally important to a lot of people, if not because said people are direct consumers of the alleged benefits then because they paid for them. We keep paying and they keep ballsing it up, and everything we know about the theory of government (in an economic sense) tells us this is a design flaw and, ergo, not something that can be expected to self-resolve.
And it's not that they're totally inept all the time (although ineptitude is a recurring theme) - sometimes they do get good stuff done. The much harder question to answer in these cases is how efficient was it? How much tax money was shat away in admin/stupid compliance BS, and is all that wasteful pomp really a necessary price to pay for democracy? Could there be another way?
It's all very contentious, high-stakes stuff and if you disagree strongly enough with it all to wilfully not comply you'll soon find the whole system is propped up on various threats: they'll take your money, your home, your pride - they may even take you. And they'll do it all under a banner of moral righteousness that turns it into you vs. "society". How could that ever not be a talking point?
I talk about it most days in one way or another, and not because I'm some horrible right-wing fanatic who wants to wipe my ass with money and glee in the suffering of others, but because it's an incredibly fragile, wasteful system that perpetually sits on a knife edge and I suspect does just as much harm as good. It's misguided in so many ways and so, so hard to change that all I can do is moan about it and hope that my moans contribute in some way to getting the volume of the debate loud enough to think that something could change.
So, yeah, that's why I think it keeps coming up.
The incredibly complex and expensive purchasing/hiring system that seems to fail every single time exists because we don't trust government employees to spend our money. Perhaps we shouldn't. But you cannot deny that the system designed to prevent improper spending is the cause of a hell of a lot of improper spending.
Government agencies are often legally bound to purchase from the lowest bidder even when they know the product will be "technically correct" but extremely low-quality. Because the regulations and processes are so complex, it is incredibly expensive and difficult to sell to the government, meaning the few companies that actually manage to do it have higher costs and less competition. Of course prices are high.
Sure, the system is necessary to stop incompetent/malicious actors from wasting money or throwing contracts to their cronies. But it also stops a lot of smart and diligent people from getting the resources they need to complete their projects.
If you want government agencies to be able to purchase high-quality goods and services at reasonable prices, then you're going to need to loosen the chains. And that probably means some fraud. And that probably means some agencies buying slightly nicer stuff than the absolute cheapest thing that would work on paper. But the overall effectiveness of government might end up higher, and its cost much lower, if you were to eliminate the "government services" industry and open it to the wider market.
I think this largely misses the mark, for several reasons.
1) The problem with government isn't ineptitude. At least the federal government is probably average to above-average in terms of competency. But what's the competency level of the IT folks in your average Big Corp where IT isn't a front office department? It's not very good. Take those not very good IT folks and then make them do projects 10x the size and complexity of what your typical Big Corp handles, in areas where the tolerance for error is nil (health care, benefits), and political concerns make things like iteration and building MVP's intractable.
2) It's easy to talk about "admin/compliance BS" because you don't have to point to any actual procedure that you might have to defend as being BS. It's not clear to me that the federal government has any more internal red tape than your typical Big Corp.
I spent a short time at the FCC, after working as an engineer in the wireless sector, and it was pretty eye-opening. On one hand, it was quite lean and efficient for what it is tasked with dealing with. At the top it's five commissioners, a few very experienced policy advisors, and a rotating staff of interns doing grunt work. On the other hand, the necessary complexity is staggering. For every decision, there are dozens of stakeholders. Things that were no-brainers from a purely engineering point of view became multi-facted problems combining social justice issues and economic issues.
I think engineers, software engineers in particular, confronted with complexity, seek to eliminate it. Something is hard? Do less. That's usually just not an option with government.
I work for a large state government. Plenty of Dilbert moments, but important stuff gets done. There are people all over the place that have dedicated their life to what they do and are masters of their craft or policy area. In IT, we have plenty of crap, but a few things that are world class and make a real difference in people's lives.
I've worked for small companies where the "friends & family plan" ruined key aspects of the business/workplace. I've also seen large companies with all manner of incompetence and bad behavior -- one bank leader I know actually built a business unit with no useful purpose... specifically for the purpose of having chum for a layoff.
As it turns out, the current rules are actually very effective if what you want is a system tailor made to produce large and profitable overages after any danger of losing a contract to a competing contractor has been mitigated.
"Trailblazer was a United States National Security Agency (NSA) program intended to develop a capability to analyze data carried on communications networks like the Internet.
In 2005, NSA director Michael Hayden told a Senate hearing that the Trailblazer program was several hundred million dollars over budget and years behind schedule. In 2006 the program was shut down, after having cost billions of US Dollars. Several anonymous NSA sources told Hosenball of Newsweek later on that the project was a "wasteful failure". The new project replacing Trailblazer is called Turbulence." - http://en.wikipedia.org/wiki/Trailblazer_Project
My pet theory is that it has to do with outsourcing and the widespread belief in the private market that software development is a product just like any other that you buy on the market.
When your average intelligence agency has a million dollars to spend on a project, it will keep a house full of smart people and tell to forth and create, paying their bills and keeping their backs. We might call this the academic model, if it wasn't so far from how academia works today.
When a civilian government agency (or any large private entity for that matter as they are completely similar in this regard) have a million dollars to spend on a project they immediately start phoning the big boys: We've got a million to spend on this project, can do do it?
It is all a bit exaggerated of course, but in this made up example one will get a working, if perhaps shoddy product, and one will get a server filled with very important powerpoints while the product itself tends to be late and slow.
Or to put things in a simpler way and less of a rant: I suspect it is a lot easier to spend a million dollars on a highly specialized system, than to do the same on a very generic system anyone could build. The corollary is that the same intelligence agency would fail in the same way a private or civilian organization would when implementing a new payroll system.
For the NSA/GCHQ/etc, the actual engineering task of setting up some DPI boxes, beam-splitters, and other wiretap equipment is not particularly complicated on the engineering side[1]. The difficulty with that kind of project is access and legality. Even their massive DB in Utah, while (very) large, is probably a fairly simple schema compared to a many business DBs.
Compare that to a healthcare website, that has to implement many random features mandated by law in a way that is compatible with a giant list of "standards", medical jargons, insurance requirements of all sorts, financial regulations, to name just a few of the details. And it has to work, because lives are going to rely on it. While this kind of website might be simple in concept, the random, contradictory, poorly-specified, "design by committee" project requirements are going to be nasty.
[1] crypto may be different, but very little of what the NSA is currently doing has anything to do with crypto. They don't even need side-channel attacks when it's all in plaintext... sigh
So all ye folks can start disguising your packets as BitTorrent porn downloads now :)
1. http://motherboard.vice.com/read/nsa-paranoia-has-fanned-the...
https://www.schneier.com/blog/archives/2014/12/putting_nsagc...
I wonder if NSA and GHCQ are changing program code names as they become compromised, or if at some point they just changed all code names regardless. I suspect the latter. If that is the case, I imagine it was probably a bureaucratic nightmare of biblical proportions.