Hacker News Reader

Comment by tw1010 | Hacker News Reader

This is such a fun observation. Physicists originally used differential equations to describe large scale, yet fundamentally atomistic, behaviour because, I assume, that was easier to work with using pen and paper than by using some more discrete, computational, atom-by-atom model. And then computers were invented, but we're still stuck with these continuous, legacy, models. Almost no-one questions the fact that the original motivation for using the differential equation technique is no longer appropriate given that the underlying technology has shifted underneath the feet of the modellers.

View on HN

Well, we still don't have the computational power to do things atom-by-atom, even in small things like modeling cells, let alone something like simulating wind shear on a plane or predicting weather patterns.

It's not just computational power but memory. There's something like 1E14 atoms in a cell. Storing the x,y,z position with 4 bytes for each dimension (though you'll probably want 8 bytes) comes out to 1.2E15 bytes = 1.2 petabytes of information. Add in more fields for velocity and other pieces of state and you're talking about something that would only fit in working memory in some of our largest machines.

Here is a billion particle model (e9) which is a long way from atomic scale simulation (e14+), but not outside the realm of the possible.

https://www.youtube.com/watch?v=B8mP9E75D08

We are definitely making progress. But another factor to consider, in additional to the number of particles, is the energy of the system. When going through one step of a simulation you have to compute the effects that every particle has on every other particle, but generally a particle only affects the other particles near it. You can take advantage of this fact to speed up computation with the right data structures.

However, as particles start moving around faster and faster, the distance they can travel in one time step increases, and the neighborhood of effect increases, limiting the speedup of this optimization. This matters less in larger scale simulations like of weather patterns, because you don't have to worry about air molecules zipping to the other side of the continent in one second. But it matters a lot in small scale simulations, especially in cells. As an example, an average glucose molecule in one of your cells is bouncing around at around 250 miles per hour! That's not 250 miles when scaled up, that's really 250 miles per hour. A molecule in your body is colliding with another billions of times every second. (Source: http://www.righto.com/2011/07/cells-are-very-fast-and-crowde...). At that scale and that level of activity it becomes much harder to simulate each time step.

The other side of this is cells are also really fast. Modeling even 0.01 seconds could be very useful.

Much of the action inside cells is done via proteins. It's taken a huge amount of distributed computing power just to figure out a reasonable hypothesis for how they fold (only one part of the movement) and €1.22 billion to build the XFEL and have a way to image them on the required timescales: 1-2 femtoseconds. 0.000000000000001 seconds. 0.01 seconds is an eternity, calculating that long would take an utterly enormous amount of computer time.

You can spend similar amounts of processing power (or vastly more) modeling a single atom over much shorter time scales from quantum mechanics. Protean folding while important results on fairly stable configurations and you don't need the same granularity after that point.

PS: Also of note protean folding simulations don't even simulate the water surrounding the protean and are again very simplified.

Don't be fooled by the numbers e9 is 5 orders od magnitude less than e14

64 GPU's and 91 hours for a 12 second simulation. 64,000 CPU's for 910 hours (1 month) is 4 orders of magnitude.

Unfortunately, a useful cell simulation would be far more complex than simple particle collisions and a much shorter step size. But, that's not to say different kinds of simulations can't be useful. And ASIC's or general improvement in computing power can also boost things.

we still don't have the computational power to do things atom-by-atom

That's what puzzles me about this discussion.

Of course we don't. The goal here is to model more atoms than we have atoms to model with. Until we get to quantum computers that can represent more information per atom, than information per atom we want to represent, it's simply a matter of objectively & obviously inadequate resources. You can't simulate wind shear on a plane when there's more atoms in that wind & plane than there are in the computer simulating them: every single atom being simulated contains the complete information about its state (information independent of any other atom), so simulating it will require at least one atom per atom simulated - if you don't have at least as many atoms to simulate as you are simulating, you can't achieve a complete simulation.

It is not necessarily the goal to model more atoms than we have atoms to model with. The goal can be, for example, to model a folding protein, which has fewer atoms than a CPU.

And one might be happy to spend a month computing the folding while in the cell it takes a few seconds.

That may be true of digital computers, but it's not necessarily true of analog ones.

> Well, we still don't have the computational power to do things atom-by-atom...

We still don't have the computational power to do certain classes of continuous Navier--Stokes calculations (i.e. not atom based).

Sure, but you'd think we would shift our focus to using models explicitly tailored with that aim in mind. Using something that explicitly approximates a desired, precise, foundation. Perhaps parameterized by model accuracy. I would think taking inspiration from approximation algorithms would be a much more appropriate lens with which to model physics.

The mental model of a bunch of hard little atoms flying around, colliding, rebounding, etc is actually NOT the desired, precise, foundation for understanding how gases behave.

The better foundation is quantum mechanics. An example of a macroscopically visible difference between these foundations are the van der Waals forces.

In the case of Navier-Stokes, this was done in 1934 by Jean Leray, who showed that the so-called 'weak' solutions always exist. (A weak solution is not a function, but a more general object called a distribution which can be fuzzed out on very short distance scales.)

It's a very reasonable idea, and useful enough for computations, but it lacks the essential tension that makes Navier-Stokes a mathematical Everest.

> Well, we still don't have the computational power to do things atom-by-atom

People do atom-by-atom molecular dynamics simulations of proteins and such.

Yes, but those are simulations of classical mechanics models of molecules which are known to be very limited when it comes to atom-scale objects. We have another model that was much more useful there (Schroedinger's equation) but it is only computationally tractable for systems of few atoms. We do not have the computational power to work out the consequences of the model for proteins.

Certainly, but not at the level of an entire cell

Your assumption is frankly incorrect.

Physics is generally expressed in terms of differential equations. This is not due to their analytical tractability - as anyone who has attempted to solve PDEs before will know, most (nearly all) differential equations do not yield to analytical solution. Perhaps you think that quantum mechanics demands a discretized view of reality. This would be a complete misunderstanding of quantum mechanics, and physics in general.

It's unclear whether atomic-level data would be useful in these sorts of calculations - the input data isn't known to sufficient precision. For more, check out the last two paragraphs from the Feynman Lectures Vol III Chapter 2:

http://www.feynmanlectures.caltech.edu/III_02.html

A similar sort of mechanism led to the discover of chaotic behavior in weather models - checkpointing results at a precision slightly lower than the machine's internal precision caused simulations that were resumed from the checkpoint to deviate rapidly.

Human limitations is one reason, but there's something much deeper going on here.

Navier and Stokes worked before we were sure that atoms existed, certainly before we had any idea of how many there were. Nevertheless they were able to write down useful theories for describing fluids. This is how all of science works. The things about which we are totally ignorant are much smaller today, of course... but useful theories of any set of phenomena always omit a great many things we do in fact know about.

From these theories, we can understand what's going on, and use this to extrapolate to things we have not seen yet. An atom-by-atom computational model would (in some sense) be no more useful than what we had before N-S, just blind experiment. To try out any given swirl of smoke etc. we can equally well walk next door to the lab and videotape it... but this doesn't help us imagine what else might be possible. The "blowup scenario" discussed is an example of this kind of imagining.

> The things about which we are totally ignorant are much smaller today, of course

One would have to disagree with this idea. The more we study the universe around us, the less we actually know. As a somewhat philosophical point, there are many times when our mental (mathematical) models get in the road of understanding. Very often, people believe that because we have a model that works and appears to give good predictive results about some phenomena then we understand the how and the what (and even the why) of those phenomena.

When this happens, we get into a situation where alternative models are actually discouraged. If one looks the the history of the 19th, 20th and 21st centuries, one can see that more and vaster avenues of investigation have arisen as time passes. Our increasing knowledge is continuing to be shown as ever smaller in relation to what we are now seeing.

No theory is ever complete, nor is it ever accurate to the extent that it describes the reality of the universe around us. All theories make those simplifying assumptions that when taken too far lead into inaccurately describing and predicting what we should see. Too often people get enamoured by the beauty of the mathematics and forget it is only an attempt at reflecting reality.

Mathematics is a magnificent and useful tool, but it is a foolish master. Too often we forget that.

Theories and models help us gain some understanding of the nature of the universe around us. This understanding, however, is always subject to change, no matter how "perfect" the theory may appear to be. There are too many scientists, both theoretical and practical, who are so infatuated and enamoured with their current models that they have forgotten that the models are approximations only and are subject to change or even overturning.

"much smaller today" meaning literally smaller: Navier didn't know about atoms, 10^-10 m, but now we know about things at 10^-18 m or so.

Sure, our awareness of how much we don't know has grown over time.

The point I was trying to make is that theoretical models (like N-S) not only don't have to be perfect to be useful, but more, are useful precisely because they are not complete. By ignoring irrelevant details we get theory, not just simulation.

I don't disagree with the idea that a model doesn't have to be perfect to be useful. But I do disagree with the prevalent idea that such theories are the "truth" when they are not.

Understanding that a model or theory is useful even when we ignore certain aspects of reality is quite different to the often displayed belief that a specific theory is "gospel" even in the face of anomalies and discrepancies of the real world compared with prediction. Too much of the "theoretical physics" genre (word specifically chosen) is based on the idea that mathematics is the means of finding the "truth".

As I said above, mathematics is a wonderful and useful tool, but it is not a good master. It provides a possible insight into what is going on. However, those insights are not "truth" as such. I have been doing a review of my old mathematics texts for scientists and engineers, as well as other resources. It is interesting that all of them talk of and demonstrate that all the mathematical models are simplified and incomplete. Yet, if one raises the various problems with the various models in use today, one is shouted down. This does not bode well for our advancement in understanding of the universe around us.

It's certainly possible to find such views in theoretical physics. But typically not from very serious people.

In my understanding the big shift was the understanding of renormalisation, Kadanoff and Wilson, around 1970. This took airy ideas about useful approximation and turned them into serious tools, which are both useful for everyday things and illuminating about why any of it works.

You are missing the distinction between a numerical solution to a set of equations and an analytical one. Usually, the analytical solution gives you the dynamics of a system at any point in time and space given a set of parameters.

For numerical solutions you have to run each individual set of parameters to find the corresponding values in time and space (not even considering stochastic equations). This is very computationally expensive.

This is the value in solving these things analytical — hence the prize.

Sorry this should read: “Usually, the analytical solution gives you the dynamics of a system at any point in time and space given ANY set of parameters.”

The “Any” is the value here for the analytical solution.

> computational, atom-by-atom model

Remember that as long as computers continue to be made atoms, they aren't going to do atom-by-atom simulations, unless these simulated systems are much smaller than the computer itself or other are simplifying assumptions that can be made.

Reply on news.ycombinator.com