Why NASA Projects Miss Deadlines and Blow Budgets
spectrum.ieee.org
spectrum.ieee.org
There's something else hidden behind this statement: NASA managers have also come to realize that proposals that under-estimate budgets are likely to look more attractive than proposals that accurately or over-estimate them. As a result, the equilibrium shifts and a certain percentage gets taken off the estimation for everything.
Gates did a lot, canceling Cold War era projects such as the F-22 rather than throw good money after bad, especially with the U.S. fighting two insurgencies (for which advanced fighter planes were a waste). His successor Ash Carter refocused the Defense Department on planning and developing for future warfare, especially in the 'cyber' domain.
I have also seen people getting penalized for delivering early because their estimate wasn't correct.
We have a benefits tracking requirement, you put in a 3 gets you 9 case (borderline fundable - probably too small and not sufficient payback, but you might get it) and they go back and ask for another 7 over 2 years and you will leave on the ask. If you put 3 gets you 30 and then ask for another 7 you may well leave on the ask, you may get to stay for the realisation of the 30, but then you go. Probably you go on the ask because someone else will be parachuted into the project (if it's believed that the 30 really is on the table).
But when I was there it was already being worked around... managers would create lots of smaller projects that delivered cross-functional behaviour to multiple projects and allowed them to externalise and hide the overages in what might appear to only be dependency lines to existing internal projects. The managers of multiple projects would collude to sponsor each others externalities so that they all had a layer of protection from being fired, obscuring what the costs were actually contributing to.
The reason I was there was precisely to build a revenue recognition type system for their project management system to try and identify which projects had asked for some other project to start.
But after that I worked at a UK govt department and they'd become so good at tracking who had been first to demand the creation of another project that their problem became one of legacy... no-one could sponsor even upgrading their long obsolete software or hardware, because even though it was a thing that benefited everyone, the project that demanded it was going to be the one that paid for it (literally paying for the certification, hardware, software, training, etc) which meant if you were the team that said "We can't support this on IE6" then you had to pay to replace laptops, OS, to get onto a later version.
My conclusion... if we don't embrace a pragmatic reality of what things cost, if we try and manage the costs tightly... the side effects usually have unintended consequences that are worse than facing or embracing the original costs.
Yes, but sometimes there is competition between projects. In that case it may pay to lie about the actual cost.
If so, I would see only two rational approaches: find another job, or hugely overestimate, and, unless things go very wrong, twiddle your thumbs for most of the time (or think of/work on a side project or study)
It’s very hard to estimate even the simplest ‘projects’ within 10%. For example, try estimating how long your trip home or your next bathroom break or coffee machine visit will take, and see how often you get within 10% (if you say “5 minutes” that gets you 30 seconds either way)
Underestimating on the contrary leads you to not getting funding for the projects you are responsible for, and that is a nice way to not get a promotion.
As in Python, it is easier to ask for forgiveness than permission.
Expense always gets pushed out as far as possible. R&D costing too much? Push it to CAPEX. CAPEX costing too much? Push it to OPEX.
Unfortunately humans are fantastic at being able to create systems where by we can easily defer anything that doesn't suit us right now.
This being said, I have no idea how NASA capitalize their assets. Accountants being accountants leads me to believe it's either squeezed into the same way as everything else, or there's some esoteric standard invented just for them.
Once you've successfully become known as a jobs creator, then your job is to spend as much as possible. You can't be looking at the efficiency of getting the program completed - you have to be looking at the efficiency of spending taxpayer money as efficiently as possible in congressional districts. The less work performed per dollar, the more efficient you are in getting the money into the economy. Hiring an engineer at $300/hour instead of $100/hour - why risk getting the cheap guy when there is no benefit to you?
Efficiency in government programs has a very different meaning than in other programs.
[1] https://www.nasa.gov/feature/release-of-revision-to-the-nasa...
Can any large organization reliably and accurately budget, schedule, and then implement large, complex projects? Which ones? How do they do it?
Could it be that NASA takes on the toughest, most daring and most technically complex projects around?
On the medical device projects I work on delays, overruns and issues come into play often. I can only imagine if the medical device I was working on not only had to not injure the patient, but do it in a vacuum, survive the radiation and survive launch!
The U.S. military, including the Navy, makes plenty of headlines for over-budget, over-schedule projects. For the Navy look at their highest profile recent projects: Ford-class aircraft carriers, Zumwalt destroyers, and, IIRC, the LCS.
That said, I know nothing about PERT.
Yes, it's fairly doable to set the project budget and schedule. Most organizations can do that fairly accurately.
The thing is that most people don't understand what a budget is. For instance, when a project is given 2M dollars and a year. It doesn't mean that the project will costs 2M dollars and be completed in a year, not at all. It means that there will be a few people allocated on it, with maybe some hardware, and we will see what we get at the end of the year.
But in fact it could also be the opposite of the fixed-cost fallacy. Project X is worth $1 billion, and is funded for $1 billion. A year later, the $1 billion is spent, but another $1 billion is needed to finish. At this point, your choice to spend $0 and get nothing or spend $1 billion and get the project done. The choice is obvious; we've already decided the project is worth $1 billion.
The problem is that a year later you face the same choice again. And then again. Provided the marginal increment is always $1 billion, you'll always agree to it, because the project is worth that.
Once you start noticing that budgets/timelines are slipping you can start revising your estimate. If revised estimates don't start converging, you basically don't actually know how much the project will cost, and saying it will cost another billion is just straight lying.
[1] https://en.wikipedia.org/wiki/James_Webb_Space_Telescope#Cos...
You're right, but this is the kind of management that the U.S. Constitution requires. The Executive branch "runs" the program, but the people allocating money (Congress) are only asked for input when the cash box is empty.
In a smaller (say, pre-Civil War-sized) country with a much much higher (constituent/representative) ratio, and with far fewer things for the Federal government to keep track of, Congress could reasonably exercise oversight of all or most of the things that they had appropriated money for. When 535 people are trying to oversee the largest, most expensive, most geographically distributed organization ever, in the history of the world, it all breaks down.
The best a legislator can reasonably do is, when somebody comes to Congress for a top-up on their bank account, to farm out a staffer to read some reports about a project and make a suggestion about whether it should get more money or get cut off. Most of the millions of line items in the Federal budget don't get even this much attention.
Congress can't manage it, but in an ideal world they could demand good management practices and refuse to fund things that go over budget without evidence of good management.
Imagine that your car doesn't start and you have it towed to a 100% trustworthy mechanic. He says that he knows what is wrong and it will cost you $500. You need the car, and $500 is far less than what it would cost you to replace the car. You authorize the mechanic to fix what he thinks is wrong for $500.
He calls you the next day to say he did the $500 worth of work but, just as he was finishing, he found something else wrong. The car won't work at all without this new work. It also costs $500. Because it is still worth more $500 to you to have a working car, you authorize this work as well.
This can continue indefinitely, even far past the point where you could have just bought a new car with the money you spent fixing this one. At every point along the way, the most prudent-, rational-seeming thing to do was to spend the $500, but this ultimately leads you to ruin: no working car, and your savings totally depleted.
This outcome is similar to that in the (also artificial) circumstances of a dollar auction (https://en.wikipedia.org/wiki/Dollar_auction).
Still, many things is real life approximate this dynamic, if not exactly, then in part. It's valuable to recognize the pattern so that you can recognize and short-circuit it in situations where you would otherwise "rational" yourself into a corner. In a dollar auction, you can pick the optimal move, which is not to play at all. In certain real-life circumstances, that isn't really an option.
This game illustrates the limits of blind cost-benefit analysis, but that is not the entirety of rational thought.
My brother's a Navy officer, and described to me an Air Force base he visited. They'd gotten a certain amount of money, and spent a lot of it on luxurious accommodations including a large swimming pool. They ran out of money without having gotten around to building a runway. Of course they got another round of funding for that.
One example: Ares (https://en.wikipedia.org/wiki/Ares_I?wprov=sfla1). Originally sold as reusing the shuttle's SSRBs, it was too heavy. So they said, "we'll just add another segment to the SSRB" and didn't mention that that would mean a complete redesign and effectively a new booster.
I work as a NASA contractor on enterprise software and see the same issues: things are sold as new, exciting, cutting edge, fast, and easy. Then they turn into a fiasco.
See this talk by Dan Rasky (inventor of PICA heatshield) for a perspective from the inside: https://www.youtube.com/watch?v=g3gzwMJWa5w
See almost any talk by Robert Zubrin for a perspective from outside, for example: https://www.youtube.com/watch?v=EaNKtIY6JpQ
And people could die
https://reason.com/archives/2012/01/26/how-much-is-an-astron...
Arguably it shouldn't even have been attempted - the sheer stupidity of the whole idea of having a single large solid rocket propelling you up without the possibility of being throttled down and with the chance of having burning solid fuel raining on your parachute in case you needed to bail out was probably swept under the rug too many times for comfort.
And that clusterfuck was stopped before it'd be able to kill anyone. In the meantime, it produced some knowledge on large solid fuel 1st stages that may someday compete with SpaceX's tech.
It won't, solid rockets are inherently limited in their safety (for the reasons you pointed out), fuel efficiency and ability to be rapidly reused.
The fuel efficiency will never be that of kerolox, methalox or hydrolox engines [1].
Landing a solid the way Falcon 9 lands is impossible (there is no engine on the way back, and if you can shut it down safely and then relight it then you can't throttle it), and when you recover the parts of the rocket that are not fuel, "refueling" solid rocket is basically re-manufacturing (the STS boosters were re-manufactured, without any economy).
In general, the use of solid boosters for crewed NASA missions is a folly that was mandated by congress to funnel money into ATK (manufacturer of many military solid rockets and explosives from hand held to ICBM/SLBM, now Northrop Grumman). Ares I, Liberty and OmegA, and the SLS boosters, are just more of the same.
1 - https://en.wikipedia.org/wiki/Solid-propellant_rocket#Perfor...
It makes sense if refueling is just filling them with fuel in an automated way (think a giant extrusion device printing layers of fuel inside a rotating empty rocket). That way, the cost of flying a rocket becomes downtime+fuel.
Maybe if SpaceX starts sending people to Mars at 1/100 the cost NASA plans to, congress will finally end the NASA manned space program.
This is small compared to some other delays, of course.
(Perhaps this forgetting illustrates the point of the article.)
Looks like nothing we didn't knew. NASA wants very high success rates on missions which leads to spending enormous sums to drive down probability of failures. On the other hand, Musk can blow up rockets dozens of times and keep moving at a fraction of the cost. The second biggest factor is simply the unknowns. It's not that we have launched giant telescopes in space dozen times and have became very good at predicting costs.
The NASA never really was about space travel, because then they would have kept the Apollo tech and liquid rockets. But military needed only solid fuel rockets so they switched and developed the shuttle. And now they have no proper tech anymore at all.
Yes, NASA HQ is in Washington and always has been. On the other hand, the centers have always had a lot of autonomy; moreso in the past, and the fight to coalesce overlapping facilities is messy. (Source: Work for the NASA Enterprise Applications Competency Center (yes, that's a thing) and get to see the sausage being made.)
JPL is one fun example. It's really only loosly associated with the rest of NASA. And it does most of the unmanned, science flight programs, using DOD launch vehicles.
That goes back to the early days of NASA following Sputnik and the Explorer I vs von Braun/MSFC/Alabama teams. MSFC has been the center of launch system development since, for the manned side of things. And the shuttle, in retrospect, seems not to have been a great idea.
Tl;dr: To paraphrase Ghandi, NASA organization might be a really good idea.