Comment by woodruffw | Hacker News Reader

Comment by woodruffw | Hacker News Reader

Parent Full thread

A bit that stood out to me:

> This "Intelligence Community Desktop Environment" (IC DTE) was conceived in 2012 as a single, identical platform for the US Intelligence Community. As such it's the heart of a huge modernization project called Intelligence Community IT Enterprise (IC ITE), under which data will be stored and processed at the Commercial Cloud Services (C2S) managed by the CIA and the IC GovCloud managed by the NSA.

> The implementation of the DTE was managed by the Joint Program Management Office (JPMO) led by DIA and NGA, while the software system was built by BAE Systems under a $300 million contract for five years.

> However, in 2018, John Sherman, chief information officer of the Intelligence Community, said they had come to the realization that it no longer made sense to deliver a standard capability to every agency and user given the differing architectures, security requirements and mission needs.

…so we paid BAE a third of a billion dollars to build a software platform that we then threw away in less than a decade, all because of an unrealistic original goal (“a single platform for the US IC”).

Unfortunately this is common throughout the DoD, where the longstanding assumption is that it is possible to have a single enterprise solution for all X, and that doing this will be much more effective and efficient overall.

I don't think it's because the people involved are completely insane; they see that IT projects in DoD are expensive, and the expense is seemingly due to government/DoD overhead not intrinsically associated to solving the software development problem. So if you could increase the amount of software you can deliver with the same overhead, it should be more efficient, right?

Of course, it's nearly impossible to make this work, but the real solution (significantly reduce the overhead so that it's again efficient to build smaller solutions and iterate improvements from there) is impossible to imagine in DoD.

Surprisingly enough you see this in other security driven, slow moving (static) industries.

Just look at industrial controls. At the base you can say “air gap the system and it’s pretty secure.” Then you add on reliability, maintainability, optimization, etc. The people responsible for maintaining the control system are not IT, so they deploy a concept OT (operations technology). Now OT and IT have to play nice to deliver business solutions that make everyone feel warm and fuzzy about stability and uptime, but in reality the overhead and complication actually diminish returns usually.

The clash between the two often results in an endless cycle of trying to achieve security and efficiency, but since the two groups are beholden to different incentives they are regularly not aligned.

This often leads to reimplementing solutions regularly in an endless churn. I’ve worked a couple times on projects to replace solutions that have not been fully developed.

I’ve been in a leadership position for smaller consolidation projects / standardization projects outside of DoD, but pretty large.

The issue is all answers are correct. A lot of the differences between the “standard” and whatever is out there come down to politics - whatever is being done somewhere isn’t necessarily there by design, it’s just there.

Usually, 80% of problems are solved by the big consolidation play. The 20% is the problem, because those are either complex technical problems or intractable political problems. Nobody can get a budget justified to spend more money on 20% of the environment, and the the people running the consolidation project push until they declare victory and get promoted.

As recently as 2018, it was actually working really well (for agencies it rolled out to). Without going into too much specifics, the technical challenges were more or less solved. There were some messy issues to work through still, but there are institutional and even practical barriers that needed to be overcome. When DNI Clapper left/after the administration change, ICITE presumably lost its champion(s) and new political appointees wanted to make their mark. Such is life inside the beltway, for better or worse.

Edit: slight edit - I will say the need for ICITE in its original conceived form has diminished somewhat as services have moved to shared cloud services. It’s not all doom and gloom.

It wasn’t thrown away. It’s going through a new iteration based on newly identified requirements. There’s a link in the article with more details.

https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...

"newly identified requirements" is the most predictable possible thing in DoD software writing though, as it is in other industries. This is why it is so important to reduce the amount of time it takes to go from identifying requirements to shipping software to product that starts to meet those requirements, as pointed out by the Defense Innovation Board's Software Acquisition Practices report: https://media.defense.gov/2019/May/01/2002126690/-1/-1/0/SWA...

To be fair, it was five years later. A lot can happen to organizational processes in that time. As organizational processes change (or new ones come about) so will the software that serves those processes.

> the most predictable possible thing in DoD software buying though,

FTFY

And if it had worked as they thought it would, it would have been totally worth it.

It’s a different level of money altogether, but 300M is a small amount of 715 BN dollars in the DoD budget planned for FY 2022.

But when you’re operating at that scale, 300M is almost in the “let’s try it and see if it works”.

DNI had 60+ BN in appropriations for FY 2020.

I'm plenty familiar with the DoD's scale -- I worked at an IC contractor for a few years, and much of my current work is funded by the DoD.

The problem isn't the amount of money itself, it's that it can't work and never will: too many stakeholders ruin normal and simple software projects, and the DoD is that at its absolute worst.

Asking all of the IC agencies to settle on something common is like asking all of the construction workers at a site to get their tools from a common toolbox: everything is going to be in there, but it's going to be a jumbled mess of different priorities that everybody resents.

Except it is working and another commenter pulled a link from the same document that describes the next iteration of the exact same project.

https://www.c4isrnet.com/show-reporter/dodiis/2018/08/14/the...

"Iteration" can mean just about anything in DoD-speak.

Barring further releases of information (or a HN reader being willing to do a little bit of leaking), I don't think anybody can confidently assert that the $300M spent on DTE will be meaningfully reused in the next "iteration."

> a jumbled mess of different priorities that everybody resents

Sounds par for the course. Link to Pentagon Wars:

https://www.youtube.com/watch?v=aXQ2lO3ieBA

I hear you, and putting the scale there was something that I wanted to check and why I commented. At that scale it’s almost like one of Google’s moon shots…

But yeah - joint programs are fraught with problems and dynamics (tragedy of the commons is one major one).

> …so we paid BAE a third of a billion dollars to build a software platform that we then threw away in less than a decade, all because of an unrealistic original goal (“a single platform for the US IC”).

In their defense, that "unrealistic original goal" is a mistake all kinds of enterprises make, because at a high level there are obvious benefits and the problems are harder to see because they're at a lower level.

Thanks for the IRC bot!

Reply on news.ycombinator.com