Hacker News Reader

Comment by drzaiusapelord | Hacker News Reader

Stupid question perhaps, but don't we have the ability to 100% simulate the rocket physics and the code it runs on in a simulation environment? Or are they just punching out code, shoving it onto rocket controllers, and testing in the real world?

I'm curious as to the QA system used here. I imagine with proper simulation this should have been catchable. I wonder if SpaceX's low cost approach means cutting certain corners and situations like these where catchable issues make it into the wild because of the difficulty of rocketry in general with the added difficulty of cheap spaceflight tacked on.

I really hope they didn't just find themselves in a STS-51-L moment where it'll take months to truly iron out the root issues. Thank goodness there was no loss of life and SpaceX's stack isn't man rated yet.

View on HN

Rocket physics are actually relatively trivial and, yes, can be fully simulated (and are). Rocket plumbing is always the hard part. Perhaps the threads on some connector got stripped as it was screwed together. Perhaps there was a manufacturing flaw or unexpected bit of corrosion in a particularly vital bolt. Perhaps a bit of contamination in a fuel line caused something catalytic to happen. Perhaps a bit of excess H20 condensation in a LOX valve caused an ice dam to form and the supply line to over-pressurise. These are the sort of problems which bring rockets down: really damned complicated plumbing problems.

I guess my point is, can we do a full simulation of every screw, material, plumbing, liquid dynamics, weather dynamics, etc and augment those with known fail scenarios and other fuzzy data to build out a real world KSP that predicts fails reliably? We should understand how things like corrosion and condensation work on a rocket engine. Considering the low cost of incredible amounts of CPU power, granular level simulation is possible on a certain level today if someone wanted to create it. We certainly see this kind of thing with stealth technology, where we can simulate every permutation of near every radar photon hitting the various surfaces of planes with various materials, scenarios, temperatures, etc.

I imagine this level of simulation might not be entirely feasible yet. Maybe for the lack of trying or budget. In a growth industry or one powered by both commercial and technical pressures, it may be difficult to sit down and build something like this out. From a more practical point of view, it may make sense to just let things explode than spend years running expensive simulations instead of building things, launching, and collecting paychecks.

I think a critical part of his point is that what you build and what you model are neccessarily two different things. You can totally simulate it, and then when the rocket inevitably fails, you can go "aha! This bit deviated from the simulation!" But you can't feed that forward beforehand to prevent the failure, since you can't expect random acts of poor workmanship or crafting. You can only prepare and hope (and you can do that with shockingly high confidence - but rockets are more than equally shockingly complex.)

Part of what makes the idea of bringing the first stage back to the pad so important is that we so rarely get to use the same engines multiple times in the field (where all the really nasty reality checks are done). Being able to reuse stages allows us to far better model how they will perform in the future. Otherwise, we're using test beds to feed parameters into sims to inform our launches; it's good practice, but more physical evidence is always better.

This.

Engineering is what hopefully guides reality up the correct branch of a theoretically possible tree.

You can simulate most of each one of those branches. But what are you going to do with a million simulation results? How does that guide your course of action? What do you do differently?

If this was an engineering or assembly defect, the answer is always going to be "Don't do that next time." If it was a design defect, then the part wasn't simulated (unlikely) or our understanding of how it operated in this design was incomplete (more likely).

The trick with rocket science is that the design tolerances are by necessity very tight. Physics dictates this with chemical propulsion. Every part you over-engineer in a weight-increasing way decreases the weight available for payload. And there isn't very much weight there to start with...

The issue is that rocket physics (essentially Newton's laws of motion) are for the most part first- and second- order, linear, ordinary differential equations with exact solutions. The plumbing, on the other hand, is governed by fluid dynamics, heat transfer, and the like. These are systems of complicated partial differential equations without exact solutions, requiring numeric methods.

>I guess my point is, can we do a full simulation of every screw, material, plumbing, liquid dynamics, weather dynamics, etc and augment those with known fail scenarios and other fuzzy data to build out a real world KSP that predicts fails reliably?

Frankly, no. You're wildly underestimating the complexity of the system.

The radar technique is possible because of the trivially-parallizable finite-difference time domain method, which most certainly does not simulate every photon (this may have been a rhetorical flourish on the part of wherever you learned this; FDTD curiously has zero error term). Nor would it simulate a wave hitting from every direction; each of those would be a separate run of the simulation.

no, we can't. we currently do not have the CPU power to do molecular simulations of fluids and solids (beyond the grain level) at large scales, that's why we still have supercomputers working at simplified versions. additionally, we don't have 100% fidelity in knowing environmental conditions, what the weather is doing at that exact point, the exact velocity/temp field of the atmosphere (which is constantly changing) at the molecular scale, which would be essential for a 1:1 simulation.

https://en.wikipedia.org/wiki/Direct_numerical_simulation

A lot of simulation already happens but simulating every imaginable scenario is impossible.

> I imagine with proper simulation this should have been catchable.

We don't know in this specific instance, but generally this is just fundamentally not true. You're way over-estimating the abilities of computers and way under-estimating the complexity of these systems.

I haven't been down voting your comments, but if I had to guess as to why other people are, it's because comments like this are typical of a certain kind of arrogance. I think we've all had that moment where we think "what, that sounds easy!" about someone else's problems, only to realise when we try to solve it ourselves we're suddenly confronted with that problem's true complexity. These are very smart people working on these problems, and your comments come across as if you're suggesting they've been negligent–while that could be true, it wouldn't be my first guess. Rocket science is hard, and things go wrong, and no amount of computing can change that.

I don't think we have the ability to 100% simulate anything - I don't doubt for a second that a lot of this stuff is being simulated already, and I think you may be severely overestimating just how useful any doable simulation is in terms of catching a bug like this one.

You have literally hundreds of systems working in concert and tied to more hundreds of physical components coming under extreme temperature and pressure conditions, some of which can interact in the weirdest and most unexpected ways - certainly not ones you'd always think to model. The chances that any one of those does something unexpected is not low, and the chances that it cascades into a much larger failure is non-significant.

edit: It's also sometimes a human problem - thousands of people working on this together, and all sorts of different incentives. Here's a famous example of a failure, and the PR kerfuffle that ensued: https://en.wikipedia.org/wiki/Rogers_Commission_Report

Quoth Feynman:

"It appears that there are enormous differences of opinion as to the probability of a failure with loss of vehicle and of human life. The estimates range from roughly 1 in 100 to 1 in 100,000. The higher figures come from the working engineers, and the very low figures from management. What are the causes and consequences of this lack of agreement? Since 1 part in 100,000 would imply that one could put a Shuttle up each day for 300 years expecting to lose only one, we could properly ask "What is the cause of management's fantastic faith in the machinery? .. It would appear that, for whatever purpose, be it for internal or external consumption, the management of NASA exaggerates the reliability of its product, to the point of fantasy."

>You have literally hundreds of systems working in concert and tied to more hundreds of physical components coming under extreme temperature and pressure conditions

This is exactly what computers are for: doing hard stuff we can't do on paper or just by real world prototype testing. I imagine this is a hard problem, but it may be so because from a time/budget perspective it may just make financial sense to let stuff blow up now and again, than build out such a system.

I kinda see this as the difference between writing typical code versus writing code that's deterministic. The former is cheaper/faster but the latter is safer but more expensive and slower. In growth industries or when you have a strict schedule on your back, the slower approach is often ignored.

>Quoth Feynman

Feynman died when the hottest CPU was the 386. We simply have the capabilities, at least in hardware, for non-trivial simulation that during Feynman's time would have required CPU resources ridiculous to even speculate about. Safe assumption in Feynman's world (1918-1988), at least in regards to technology and engineering, may not be safe assumptions in our world. The same way our assumptions today won't make too much sense for our grandchildren. They might be bewildered by the idea that rocket fails were constant and common, the same way I'm bewildered by things like hot-days causing vapor lock to shut down old cars or, say, occasionally tuning a carburetor. We have electric gas pumps and computer controlled fuel injectors now.

edit: to reply to jacquesm. That's a pretty bold claim about O-rings. We fully understand the materials they're made of, their typical decays, etc. They're not magic. If someone wanted to make a top-down simulation that included, well, everything, it certainly seems possible to me, and while certainly not perfect, if done right, should provide positive outcomes. The real question is, what's the incentive? Spend billions and years doing this for one system (which may be old or even obsolete by the time the simulation is complete) or just accept the occasional preventable loss. Seems the latter approach just makes more sense financially, but that doesn't mean the former approach must be impossible. Many things are possible that just aren't incentivized.

Oh gosh, having worked on a fairly large vacuum system, I can tell you that o-rings are monsters. Very minor errors in dimensions can mess up the seal, and temperature/humidity/wear/elasticity and all that can subtly mess with the dimensions in crazy nonlinear ways. You can simulate the ever loving garbage out of it and an imperceptible change in composition due to an undetectible mixing error when extruding the ring can cause a seal to slightly leak. Mayhem ensues. (And most likely any attempt to directly detect it will destroy the integrity of the o-ring or take so long to render the test useless, since there are usually hundreds of o-rings (or in the thousands - o-rings are all over the place).)

I'm not even talking about jackquesm's note about the failure mode, either. Just real insidious errors in manufacturing that can't be detected in any sort of reliable, sane way. Even the Challenger's o-ring wasn't guaranteed to fail, and indeed most didn't. In fact, most of that entire o-ring didn't fail.

I've seen some really freaky things amplify what are essentially chaotic edge cases. You can certainly figure them out, but you'd never get anything done for any level of affordability in time for any ship date if you didn't just calculate risk and go ahead.

TL;DR: risk is always there because the world's imperfect. At best you just tighten the statistical confidence, but that's super hard.

Faster computers do not equate to magically better programs and/or programmer capabilities. The computers from Feynmans days could do finite element analysis and structural simulation on grids fine enough for just about all engineering work. It's the execution details that get you (such as an O-ring...). And nobody simulates the execution in a meaningful way simply because there isn't enough data to start your simulations with. These are human failures first, process failures second.

The problem with simulating failures is that there are an infinite number of them. Should you simulate the effects of omitting each individual molecule in the whole assembly? Or adding one of every possible contaminant molecule at every possible location? What about more than one? It's a combinatorial impossibility.

> but don't we have the ability to 100% simulate the rocket physics and the code it runs on in a simulation environment?

No, we don't. While we can do this very well for the "high level" physics, we can't do it very well for the "low-level" physics such as -- for one example -- the detailed effects of turbulent flows (both outside of the craft and inside of the plumbing), which are usually modeled based on averages of aggregate effects and random models because a detailed deterministic model is impractical (both, IIRC, because doing so at the level we could in theory do is too computationally expensive to do in practice, and because even our theory is pretty limited when it comes to turbulence.)

> I imagine with proper simulation this should have been catchable.

Its possible that there is some level of simulation which would have caught this (we won't know unless they figure out with enough detail what the problem was), but even if it would have been possible, it may not have been cost effective.

I recall John Carmack mentioning that they tried fluid dynamic simulation with their rocket motors. It got them a general idea of how the motor would perform, but in real life the performance of the motor was significantly affected by small changes in e.g. the quality of the welds in the combustion chamber.

The book The Martian, big-budget movie coming soon, has some pretty awesome fictional-yet-very-well-informed situations of things that were so unimaginable to have an impact on anything and still caused things to go terribly wrong. Highly recommended.

Science has yet to discover the ultimate physical laws that govern the universe so 100% simulation is still impossible even in theory. In practice high fidelity simulation is still hard. Here's a video about the state of the art in combustion chamber simulation and what it's current state is:

https://www.youtube.com/watch?v=txk-VO1hzBY

Fluid dynamics is still very hard to simulate, and that's where rockets go wrong.

Reply on news.ycombinator.com