Hacker News Reader

Comment by aglavine | Hacker News Reader

From what I've read it is analog to Waterfall (Boeing) vs. Agile (SpaceX)

So the same except everyone at SpaceX has an extra meeting in the morning?

In fairness, the SpaceX people likely also get the pleasure of getting to work a few extra hours each day in order to ensure they complete their sprint. So that's nice...

It also seems they get the pleasure of shipping working products on schedule - something many software engineers can only dream about :-)

For some values of "on schedule".

hehehe ... true story.

I worked at Lockheed Martin as a Systems Engineer for a few years. Everything we worked on as a project was waterfall. We did annual training using the methodology and did exercises around it as well. I think it's rather commonplace for defense contractors and wouldn't be surprised if this were still the case for Boeing.

Both use waterfall to some extent.

Good point - I would guess that Boeing's process is a much longer and more extensive waterfall (i.e. one major iteration on a large scale), whereas smaller companies like SpaceX go through the waterfall for multiple shorter iterations. There are advantages and disadvantages to both approaches, I assume.

I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration. Every flight from the first flight must be flawless. Look at Spacex ... they iterate just fine for their own private fleet but the minute they start work with nasa they grind to a a slow pace.. boeing, bless their poor hearts, grew up on the nasa pork so they aren’t structured to iterate if they wanted to... like a declawed cat who has never been outdoors pretending to be fierce, they thrive on the impression that what they are doing is space exploration. And now when it’s starting to become real... oops...

> the reason nasa projects fail is because they refuse to allow for iteration

Could this be a relic of the era when we needed people in the spacecraft? As far as I'm aware, SpaceX has never had manned flights - a catastrophic explosion is momentary bad press, but they can continue on. Every time someone died in NASA it was a real tragedy. Plus every NASA launchpad explosion is more obviously "wasted" tax payer money.

I think NASA had unmanned test flights even in the early days.(https://en.wikipedia.org/wiki/List_of_Apollo_missions#Unmann...).

I think everyone would agree that manned flights should be as thoroughly tested as possible. Although SpaceX has never had manned flights, they do dock with the ISS, a failure of which could have very real consequences including possible loss of life.

The belief that an iterative, agile approach is too "fast and loose" for complex engineering projects has been throughly debunked, but for some reason continues to pervade.

Your comment comes across as if it's a forgone conclusion that agile development is adequate but it seems like there is still plenty of debate on this in terms of safety-critical systems.

"The quality control mechanisms supported by current agile processes (e.g., informal reviews, pair-programming) have not been proven to be adequate to assure users that the product is safe. In fact there is some doubt these techniques alone will be sufficient. Formal specification, rigorous test coverage, and other formal analysis and evaluation techniques included in software engineering approaches provide better, but also more expensive, mechanisms to tackle the development of safety- or business- critical software" [1]

SpaceX has shown quality control issues on their hardware processes in the past, which, from the outside looking in seems like they should have been caught.[2] I wonder if the fact that they are a relatively young organization is why these processes were lacking to begin with. It seems to be the nature of the learning process that as the 'unknown unknowns' are uncovered, processes will become more robust (and possibly cumbersome) by extension. The real question is whether they should have been unknown to begin with. I also wonder if they will start to look less like the agile upstart as they create more mature processes.

[1] https://arxiv.org/ftp/arxiv/papers/1409/1409.6600.pdf

[2] https://www.wsj.com/articles/structural-failure-likely-cause...

Reference 1 is nearly 20 years old, much has changed in those 20 years.

Reference 2 is non sequitur; nobody is suggesting that there are zero errors/failures in agile, simply that Boeing may benefit from taking a more agile approach.

Here's how iteration works: https://www.youtube.com/watch?v=bvim4rsNHkQ

I agree that the attitude regarding agile development has changed, but the point from Ref. 1 was that the topic isn't nearly as settled yet as your comment might lead one to believe. There is still considerable debate within the aerospace industry on how and when agile development is appropriate. I would be legitimately interested if you have relevant references to successful applications of agile methodologies to safety-critical human rated space software development.

Sorry if I wasn't clear enough on the point of Ref. 2. The reason that was brought up was the issue found was related to supplier quality. The fact that these checks were not in place is a bit surprising from a quality perspective and SpaceX indicated they will include additional quality oversight in the future.[1] I.e., through failures, they are adding additional processes that move them away from being an agile organization and closer towards the bureaucratic processes they are often contrasted against.

"SpaceX will implement additional hardware quality audits throughout the vehicle to further ensure all parts received perform as expected per their certification documentation."[1]

[1]https://www.spacex.com/news/2015/07/20/crs-7-investigation-u...

"At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly." - The Agile Manifesto

I'm familiar with the agile approach, but what you quoted isn't unique to agile; it can be found in any quality program that implements continuous improvement. It also slightly begs the question by using the agile manifesto as a reference to justify agile development.

What I am hoping to find is more concrete examples of successful application of agile processes to safety-critical software. For example, how can user stories be used in lieu of a traditional software requirements specification and still effectively capture all the necessary safety requirements (including those the customer may be unaware of).

I think we will really see how this plays out with starship. Just look at the number of iterations it’s gone through. I personally prefer to plan everything and think of every iteration and eventuality. But Whenever I do post mortems in my projects, I find that the most rapid learning came from putting on my knee pads and getting under the frame and getting grease under my nails and dust in my hair. Sorry for the analogies!

Oh it’s definitely because of this... that’s why I think Spacex was right to pursue a path where each flight gets considerably cheaper with reuse and to prove the platform with cargo rather than pursue space tourism. Now they have a flight proven business and can work on the hard stuff. Each time a broken timer doesn’t work for spacex, as long as the rocket is recovered, they basically lose the cost of the capsule and the fuel. Every time Boeing fails, it’s what? $100 million (I don’t have a source for that)... spacex could have iterated already and have a man in space but nasa is worried about protecting Boeing’s ego that they are slowing spacex’s cadence... maybe that’s not fair... I get that nasa also is not an agile org so it needs time to absorb and analyze findings too. Regardless, at this point, I think it’s clear that nasa is propping up a failed program at Boeing and the sooner they deal with it and SLS, the sooner we can get on to actual space exploration.

The cost of a new-built capsule is likely more than the cost of the rocket, for both companies. Though Boeing may not be out that money in this case -- the CST-100 is supposed to be reusable, so there's at least the prospect of relaunching the same capsule if this one lands safely (and they will try to get it back on the ground even if ISS rendezvous is no longer possible).

As for SpaceX's cadence, one of those NASA-mandated tests, which was thought to be low-risk beforehand, did blow up the capsule. That sort of thing makes it a little awkward to be critical of the test requirements afterward. And while I am also very much a fan of the "cheap iterations" approach up to a point, it's less appropriate once people are on the craft and their lives are at stake.

My understanding is once the capsules splash down that’s it for the capsule as a passenger ferry. Does Boeing get to reuse theirs? I lost track of the final resolution on the stand test but if I recall I don’t think that was intended to be low risk..they shook it until something broke if I recall. But even so wouldn’t you count that as an iteration?

Boeing does get to reuse theirs. SpaceX is ultimately planning to reuse them as well, but is planning a new capsule each flight for the initial contract while negotiating recertification requirements with NASA. (That's what they did for cargo dragon as well -- they're reusing those now, but it took a while.)

From 1958, when McDonnell (now Boeing) built the Mercury spacecraft, through 1969, when Apollo 11 (first stage and lunar rover built by Boeing, 3rd stage by McDonnel) landed on the Moon, Boeing rapidly iterated and was an 'outdoor cat'. NASA moved pretty quickly back then, too.

They've just gotten comfortable indoors, growing fat, lazy, and old in the decades since then.

I know your comment is the typical explanation, but to play devil's advocate, it may just be that as organizations mature they move away from "innovation" and towards "maintenance". There's a perspective that that may not exactly be a bad thing. [1]

I've heard people say SpaceX looks very much like NASA during the Apollo days, including the average age being in the mid-20s in the control rooms. Contrast that with today where NASA's average age is north of 50. What I think will be interesting is if SpaceX maintains this over the coming decades or if they will become more bureaucratic as the organization ages.

http://freakonomics.com/podcast/in-praise-of-maintenance/

> I was just about to write something similar - the reason nasa projects fail is because they refuse to allow for iteration.

If that's why they fail, do they regularly succeed in spite of it or also because of it?

That’s fair criticism... can I qualify that I meant new development? Curious, what do you see as succeeding?

I mean, they regularly build robots, strap them on huge rockets, shoot them 300,000,000 miles away to mars, land them without a scratch, and drive them around for years. That’s awesome.

Reply on news.ycombinator.com