The Dysfunctional Trillion-Dollar U.S. Fighter-Jet Program | Hacker News Reader

The Dysfunctional Trillion-Dollar U.S. Fighter-Jet Program

nytimes.com

109 pointspseudolus76 comments

> One solution favored by Winter during his recent tenure was so-called agile software development. His vision for “continuous capability development and delivery” resembles DevOps, a popular method in the private sector for quickly testing and evaluating features for new products. Coders generate software upgrades or patches in a matter of days or weeks, pass them along to users to test and then push out the update more widely if the changes are successful.

Are the rest of the details of the article, about domains I don't personally understand, equally subtly inaccurate and misleading?

"Are the rest of the details of the article, about domains I don't personally understand, equally subtly inaccurate and misleading?"

I think that's the case for most things where people who are not complete subject matter experts write about something. A lot of our own conversations about things are probably quite inaccurate in a lot of details.

In this particular case the author may even have asked back but what is the author going to do if he/she gets fed inaccurate information? I think the only blatant inaccuracy is the reference to DevOps, otherwise the description of agile is pretty OK too me.

I hope people won't fall into the usual trap where some inaccuracies in some media leads them into reading much less accurate but more extreme media which is something you often see in politics.

This is sometimes called the Gell-Mann amnesia effect.

https://www.goodreads.com/quotes/65213-briefly-stated-the-ge...

Made up by Michael Crichton, if I'm not mistaken.

Like the internets, the news is largely composed of views by people who are typically not experts in the subject domain.

A tenet of journalism is that professional practices can provide some level of truth regardless of expertise.

Note that this article will one day be cited in a DevOps wiki page by a well meaning writer far abstracted from 2019 HN. That’s the way most history is written.

Yes. The "trillion dollar" figure comes from Wikipedia, where it (a) is denominated in 2070 dollars, and (b) includes "operations and sustainment".

Is it really that bad for a general readership? I'd like to see automated testing in there. And maybe incrementally improving products. But to someone who doesn't know and doesn't care about the details of software development, it seems to capture the basic gist.

Putting aside the accuracy of the NYT's definition, what's confusing to me is...why even mention "DevOps" in the first place, as if it were relevant elsewhere in the story, or as if it were something the general readership would know enough about to use as a point of reference (which I'm assuming they don't)?

Remove the reference to "DevOps", and you have a concise enough explanation of "so-called agile software development" and a cleaner paragraph overall:

> One solution favored by Winter during his recent tenure was agile development, a popular software engineering methodology used for quickly testing and evaluating features for new products.

(Since software teams in the U.S. government, such as 18F, also use agile dev these days, so it's not particularly relevant to point out its popularity in the "private sector")

Good point. As you suggest, the definition may or may not be "OK" but it certainly doesn't get into anything around how DevOps might be different from or go beyond agile. For a NYT audience, that's a pretty decent quick definition.

The current title I'm seeing is "Inside America’s Dysfunctional Trillion-Dollar Fighter-Jet Program" which seems misleading as well; it's more of a history of press releases. Doesn't look like they interviewed anyone actually working on it aside from the fellow in charge if that.

The article describes at length the thoughts of Lt. General Christopher Bogdan, who was in charge in 2014; Vice Admiral Mat Winter, who was in charge until this past July. The reporter for this article actually works for Defense News, and the NYT article cites material from, and links to her June 12 article, which refers to "documents exclusively obtained by Defense News" which describe 13 "category 1 deficiencies":

https://www.defensenews.com/air/2019/06/12/the-pentagon-is-b...

Note that the posted article is a NYT Magazine piece, which has different editorial style and convention from the newspaper. Typically, a story this long wouldn't be written by a non-staffer in the newspaper. But it's not uncommon for the magazine to commission a writer who has written a notable article for a different publication to generalize and elaborate on it for the NYT Magazine.

I think we agree on the sourcing, just we disagree on the definition of "inside".

How does such a poorly researched article get published in the NYT? My personal blog is better researched than this.

The author reported an extensive story about the F-35's flaws based on exclusive documents in June. This submitted story revisits that work (and writes it in the style of the NYT Mag):

https://www.defensenews.com/air/2019/06/12/the-pentagon-is-b...

If you hit Publish on your personal blog three days later than originally planned, you’re unlikely to be fired over it.

It's because they don't care enough about being accurate. Maybe you're right, and that carelessness is a byproduct of speed. But, it's not like this was breaking news. They had enough time for a technical editor to have a look.

Clicks are what publishers chase now, there is only so much time a modern reporter can research before they need to publish something. They have to weigh short term revenue vs. Pulitzer prize. An article or series of articles that truly brings enough positive attention to the defense industries largest project and the players involved might be considered, but can a team focus on that in the current click driven publishing space?

> The current title I'm seeing is "Inside America’s Dysfunctional Trillion-Dollar Fighter-Jet Program" which seems misleading as well

"Dysfunctional" is also misleading. The program has produced three operational variants, including the first ever supersonic STOVL aircraft. The US and Israel have both conducted F-35 airstrikes. That is a called a "functional" (if by no means perfect) aircraft development program.

It's also encouraging to see the F-35A price falling as expected...

The comment you point out reminds me of something Margaret Hamilton said:

Also, what became apparent with Apollo – though it is not how it worked – is that it is better to define your system up front to minimise errors, rather than producing a bunch of code that then has to be corrected with patches on patches.

Source: https://www.theguardian.com/technology/2019/jul/13/margaret-...

Someone who did impressive things with method A saying they think a method B that they haven't tried is superior isn't much evidence in favor of B.

So, in other words, it is better to do waterfall development?

Indeed, it sometimes probably is, and agile generally shouldn't mean you just make it up as you go along. But I think we've generally come to recognize that, at least in many cases, being more flexible and iterative.

This, to be clear, is not in any way a knock on the many people who developed software for Apollo. It was a type of project and set of technologies that made it difficult to be as iterative as, certainly, pure software projects are today.

I think her comment is specifically around software that is critical for human life support. She is advocating for design in such cases. I posted it because I would consider the SW for the F-35 to be in this category.

It’s worth questioning whether agile SW development that we do for websites, apps, etc... is good when human life hangs on the line.

Clearly, there was never going to be any way to patch the Apollo Guidance Computer in the middle of a mission. Even if that weren't true, the development cycle was so long that iterative development was never a realistic option.

I suspect a project like the F-35 is a hybrid, with elements that are life-or-death mission critical and absolutely need to Just Work, and elements that allow for refinement - even refinement-by-failure in some cases.

Which just underlines how complex a project like this is. A distant descendant of the Apollo Guidance Computer code is likely just one of (at least) hundreds of equivalently complex routines and features in the F-35 codebase.

Absolutely. The techniques depend upon the circumstances and that's often not recognized. Or at least there's over-rotation in one direction or the other to a degree that may not be appropriate.

I would not be surprised, if 2-3 weeks from now, one of the US Presidential candidates, will be talking how they would save money (and redirect them into other programs), by decommissioning this kind of wasteful military programs.

And they will refer to this, 'well-researched piece by a 'reputable news source.

Let's see if I am right, will check in in 2-3 weeks...

Not just in this, but in virtually every newspiece you’ll run across. This can be difficult to remember, but is well worth doing.

The author is an expert in defense, specifically warplanes. She's not an expert in software engineering.

In the modern era, is it in fact possible to be an expert in the former but not the later?

That's just nytimes doing what nytimes does best. I've never found their articles to be well researched and either accidentally or purposely misleading.

News about them lately is also not in their favor. I'm sure this is just another piece (which has been covered to death by better and more accurate publications) that is intended to discredit the current administration more.

What's the error here? Using the word "DevOps" instead of "agile software development"

This is not that bad a quote.

Seems good enough for a general audience.

> There soon turned out to be an essential flaw in the grand plan for a single plane that could do everything. Design specifications demanded by one branch of the military would adversely impact the F-35’s performance in another area. “It turns out when you combine the requirements of the three services, what you end up with is the F-35, which is an aircraft that is in many ways suboptimal for what each of the services really want,” said Todd Harrison, an aerospace expert with the Center for Strategic and International Studies. “It is much more expensive than originally envisioned, and the three versions of the plane actually don’t have that much in common.”

And nobody was surprised.

I think the Navy and Air Force could have shared version with a lot in common. The Marines want VTOL and I understand that feature is important to the Marines. Unfortunately, the VTOL requirements have force compromises in the design. Without VTOL, I believe the Navy and Air Force would have ended up with a platform that was more acceptable then what they are getting.

In other words, nobody learned anything from the Space Shuttle program.

That and there was some other 1950's era fighter that they tried to make dual use and eventually the Navy convinced congress to put a stop to it. Which they could because during the cold war it wasn't just about funneling money to congressional districts.

Me I think the F35 is the 21st century equivalent of a battle ship.

I worked on the Comanche Helicopter program for a while, many years ago. The program was based on the same fundamental idea: create a helicopter that that could perform the roles of advanced armed reconnaissance and attack. The idea has a great deal of appeal. You could save on training pilots and maintenance crew. You could save on supply chain, and the cost of spare parts.

Unfortunately, coming up with a design that could effectively fill those roles was more than they could accomplish. Fortunately, the program was cancelled after spending ONLY $6.9 billion.

The question isn’t whether the compromises are bad but whether 2 or 3 different programs with the same budget as this single program would have done better. That’s possible but not at all as clear as “the F-35 is expensive and technically a compromise between too many different goals”.

I've been following the development of this plane for a while now. I remember reading in a different article that the intention of this plane is to be a sort of everything to everyone type of thing. It's meant to replace a lot of other planes that are more fit to their specific purpose and has a ton of bells and whistles. Pilots hate it. I find that striking and it further validates some principles I've developed over the years. Namely, smaller units fit to purpose that compose together to make a larger whole will defeat a single unit designed to handle everything. It also validates that management cannot force a project to be successful no matter the stakes. At best you get a half assed final product delivered on time but untested and brittle.

Too many folks here are commenting on the F-35's design as an airplane, instead of what it really is.

The F-35 is a beautifully designed trillion dollar uncuttable expenditure. If you look at how the program was designed to consume money, instead of deliver anything of value; it's a work of art!

The F-35 program could have sent every child under the age of 18 to a trade school, professional training, or college. I'm not saying we _should_ do it unconditionally.. just pointing out how we can fund an insurmountable project by not wasting money on what the big government contractors want more than our armed forces or the taxpayers

There's also the $10e12 JEDI program, ready go straight into Amazon's pocket.

Trillion dollar uncuttable programs are dime a dozen...

An interesting angle which is underreported in my view is who is responsible for extremely expensive new military aircrafts: (former) pilots, who are now High-Ranking officers. No pilot has any interest in giving up flying and switching to drones/remote aircraft which are clearly the future. My bet would be that the first country which bets 100% on autonomous aircraft will probably be China, where there will be less opposition to such a move. The western countries will follow and the F35 will most likely be the last "big" non-autonomous program (excluding aircraft for special roles).

On an unrelated note: I submitted this article 4 days ago without any discussion [1], does anyone know what the time frame for resubmission on HN nowadays is?

[1] https://news.ycombinator.com/item?id=20758128

The future is drones and so many in the air force know it. Not sure why they haven't bet fully on it. I guess there are no dog fighter drones yet but the first time a supermanuverable <10m drone destroys a 100m manned fighter well it is all over.

Supermanuverable <10m autonomous aircraft that have destroyed modern fighters already exist. They're called missiles.

Modern fighter aircraft are expensive because they are relatively large and crammed with tones of expensive avionics and sensors. Increasingly, the quality of your sensors and your ability to evade detection are more and more important while maneuverability is less important because of said missiles. Better sensors are larger and require more power to operate necessitating a larger aircraft. Also, the number of Gs an aircraft can pull is limited by both human survivability and airframe strength. This means as the size of the aircraft increases the max number of Gs it can pull will decrease. Missiles can pull 50 Gs. Even if you remove the human, no aircraft large enough to carry the necessary sensors, fuel, and weapons to be combat effective will reach even 1/3 of that because it would rip the wings off.

It's great to see this in the Fighter-Jet programmed mostly with C++, instead of Ada. I've read of rather queer flaws, such as that one mentioned involving the helmets. To its credit, C++ programmers are cheaper and more numerous, so you get what you pay for.

People will mention every catastrophic flaw Ada programs have ever had, but how many people will remember this as an example of how C++ fails at being a real language for real projects?

A thousand parts, suppliers, personnel, sensors, design sheets, and technical documents, and you first instinct is to blame the programming language?

They're building a plane, not Microsoft Excel.

A trillion is a big number. Space station big.

Personally, my expectation is that the F35 is America's $30,000 toilet seat -- financial cover for our secret UFO program of much higher performance vehicles. Building parts in 50 states is, of course, good congress fodder, but it's also a good way to make sure no one knows what they're building.

It is this lie I tell myself to continue believing in America's technological superiority :)

Yes, I had the same idea. It is so insane, overpriced and ineffective, that it must be a fake budget, used to hide a black program that will deliver the real technological advantage.

A really interesting biography which shows this is far from a new problem is Boyd: The Fighter Pilot who Changed the Art of War.

I read it a few months ago. Highly recommended.

I think Steve Blank is a big fan of this book

It seems 3 new separate aircraft would have been much better and cheaper.

It's true that the F35 seems a little beefy, compared to other aircrafts.

A single engine is quite bad for survivability.

While I can admit that vertical liftoff can be an asset if you want to respond quickly to an attack without building airstrips every 50km, it's hard to really justify building so many aircrafts with a airframe shape that allows VTOL. VTOL is a tough compromise on aircraft.

Having an aircraft that can play several roles is not a good idea. Regrouping roles is a good idea, but regrouping them all into only one is really not a good engineering decision.

That's like deciding to design a pen-lighter-laser-pointer-usb-drive-knife-screwdriver into a single device. It sounds cool, but in the end, it's better to see what can go together and what should not.

Unless the F35 has a next gen radar which allows it to see enemy aircrafts from farther, it will be bad at dogfights.

There is a design space betwen VTOL and having to build airstrips every 50km. Sweden has done it:

[1] https://en.wikipedia.org/wiki/Saab_37_Viggen

[2] https://www.youtube.com/watch?v=fBE41A9VT3Q

[3] https://en.wikipedia.org/wiki/Saab_JAS_39_Gripen

Just use some streets and prepared sheds :-)

I liked this quote from the article in that regard:

> “This is going to be the first fighter jet produced in the thousands for a very long time,” said Richard Aboulafia, an aerospace analyst with the Teal Group. “None of this is stoppable. It will be remembered, as the smoke clears, as something that worked far better than critics thought it would, but something you’d never, ever want to do again.”

Car companies develop platforms on which they build many different models. For example in VW cars you can find exactly the same parts in Golf, Touran, Passat, Tiguan and many other models.

All these 3 separate plains probably could share 90% of parts, software, avionics, that VR helmet, etc. Having fewer constraints it would also simplify the plane body design.

Just image car engineers are asked to design one car to replace off-road, city and van cars. Even Elon Musk did not come up with that idea.

> A single engine is quite bad for survivability.

True (not as bad as it once was), but is a lot cheaper over the life of the program. They were trying to get a next generation F-16 to go with the next generation F-15 (the F-22), and that didn't happen.

> A single engine is quite bad for survivability.

Are you sure about this?

https://www.quora.com/Do-twin-engine-fighter-jets-provide-hi...

From my research it looks like the same is true in civilian aviation. Statistically twin engines are more dangerous.

https://www.flyingmag.com/wrong-worry-twins-versus-singles/

Single-engine fighters are ok for the Air Force, who fly high over land. Twins are preferred by the Navy and Marines, who either fly over water, or down low and get shot at, or both.

The civil link you provided refers to "light twins" (ie. underpowered and used in GA) used in training and EMS. Not applicable to airliners, which can safely take off on one engine.

Something to think about: all of my former twin instructors have been killed flying light twins.

Four-engine airliners are actually preferred for oceanic routes for safety, but regulators now allow twins (ETOPS), for cost reasons.

Source: commercially-rated airplane pilot.

Unless the F35 has a next gen radar which allows it to see enemy aircrafts from farther, it will be bad at dogfights.

The point of seeing your enemy from afar is that you launch a missile and avoid the dogfight.

It is bad at dogfights http://blogs.reuters.com/great-debate/2014/07/14/pentagons-b...

It's really not. To put it mildly, the report that that article draws from has been subject to some criticism over the years. To put it less mildly, if you link it in any community that follows military aviation closely they will probably make fun of you[1].

Performance is generally better than F-16s in real wold scenarios due to the more powerful F135 engine and the internal weapons bay which means carried weapons don't cause additional drag as they would if carried on pylons. That second point is extremely significant. The aerodynamic effects of hanging a bunch of stuff of your wings is pretty major, especially at supersonic speeds.

Here's a few pilots commenting on differences between the F35 and other aircraft they've flown[2].

[1]https://i.kym-cdn.com/photos/images/original/001/129/519/4d0...

[2]https://www.airspacemag.com/military-aviation/f-35-faces-mos...

The enemy isn't a pilot flying something like an F-16, the enemy is a hypothetical low-cost high-maneuverability remotely controlled drone fighter which isn't limited by human tolerance for G-forces.

Or even a linked drone swarm.

On that basis, the F-35 will be obsolete against the other superpowers by around 2035 at the latest.

See my reply to the other person making that kind of claim https://news.ycombinator.com/item?id=20796007.

To expand upon that, manned aircraft aren't going away anytime soon because electronic warfare is a thing and until drones are capable of human level decision making you need a human in the loop. Drones will augment the F35's capability but against a near peer adversary you're going to need a manned aircraft at the center of every drone swarm. Especially if you're taking about small drones as those will be lacking the high powered radar which will be crucial in such engagements.

>The enemy isn't a pilot flying something like an F-16, the enemy is a hypothetical low-cost high-maneuverability remotely controlled drone fighter which isn't limited by human tolerance for G-forces.

The F-35 has internal and external mount points that are specifically designed to carry these drones.

> That's like deciding to design a pen-lighter-laser-pointer-usb-drive-knife-screwdriver into a single device. It sounds cool, but in the end, it's ...

... sorta like a smartphone

Perhaps some of this budget is part of the >$21 trillion unaccounted for & unauditable funds in classified budgets.

https://hudmissingmoney.solari.com/the-real-game-of-missing-...

I wonder if there's a system where the contractors apart of the program, which according to the news reports are so above budget / behind schedule, could submit post mortems to whichever agency audits this program. What hurdles they wish they knew going in.

This is a not a new problem. There is a classic comedy scene of the Bradley fighting vehicle's problematic design process here: https://www.youtube.com/watch?v=aXQ2lO3ieBA

Will there even be pilots in dogfights in 20 years? Won’t it be mostly autonomous by then?

It stands to reason removing the human component would be a far better engineering choice than this monstrosity.

I’m reminded of the quote, “A flying car is equally bad at being a car and an aircraft.”

I think you're badly overestimating the speed of development in defense tech. The F-35, a comparatively conventional aircraft, started development 20 years ago and look where it's at. We'd be incredibly lucky to have an even slightly functional one-off prototype of an unmanned fighter 20 years from now.

All the “sixth gen” or “next gen” programs recently started claim to be optionally manned. E.g the British/Swedish Tempest (introduction planned 2035) and the Dassault/Airbus NGF (also 2035-2040).

It’s not inconceivable that one or both of those programs will have an optionally manned fighter in active service in 20 years (2039).

It's equally conceivable that the "optionally" will be dropped as problems develop and they need to try to control costs.

I’m not very familiar with the details of what’s needed tecnically for ”optionally manned”. Maybe it’s extremely complex and expensive. My guess would be that it’s not extremely complex or expensive. UAVs have been in active duty for years so the remote piloting/link tech is already mature. Converting a current fighter to unmanned is done routinely at under $1M per hull, to create target drones. And those fighters were never designed initially to be unmanned, that’s just a hack.

Pairs well with today's discussion of multi-stakeholder coordination on managing forest fires: https://news.ycombinator.com/item?id=20783166

Reminds me of the Bradley Tank in Pentagon Wars: https://www.youtube.com/watch?v=aXQ2lO3ieBA

It still blows my mind that a trillion dollars later no american has the courage to say "so our trillion dollar plane...it doesn't fly.

US is miles ahead...someone have the courage to level.

On another level I present you the new Spanish submarine, first too heavy and now too long and obviously too late and too expensive : https://en.wikipedia.org/wiki/S-80_Plus-class_submarine#Hist...

It's not 2080. The 60 years of service aren't over yet.

Reply on news.ycombinator.com