Comment by simonh | Hacker News Reader

Comment by simonh | Hacker News Reader

Parent Full thread

I’d say it’s just a different kind of evidence. It’s obviously not on the same level as direct observation, but it can provide useful information we can use to refine our theories and home in on specific behaviours we can look for in direct observations and experiments. Saying that this information is not evidence, of any kind, seems to me to be terminological points scoring. It’s evidence for what to look for next experimentally.

I’m also annoyed by people saying things like simulation “isn’t science”. We’ll obviously it’s not all of science, we can’t run a simulation and say all right, job done, we’ve finished the science. Nobody is claiming that, it’s a complete straw man. Simulations can absolutely be part of the scientific process.

I just don’t see the point in these “well actually” rebuttals of claims never actually made by these researchers, or the author of the article. It’s the same old story every time any science news gets posted here.

It's pre-evidence.

Computer simulations of real-world theoretical phenomena are propositions of potential outcomes that need to be verified through real-world testing to determine whether our modelling correlates with, or is skewing away from, reality.

> It's pre-evidence.

No, it's pre-dictions. That's what the rest of your post says. Yes, simulations are one way of finding out what your theoretical model predicts. But evidence is the information from the real world that you compare the predictions with. There is no such thing as "pre-evidence".

> I’d say it’s just a different kind of evidence.

No, it isn't. "Evidence" means something the real world tells you when you run some kind of experiment or make some kind of observation. Simulations are not the real world.

> it can provide useful information we can use to refine our theories

"Refine" in the sense of "improve the machinery our theories use to make predictions", sure. But predictions are not evidence. Evidence is the information from the real world that you compare the predictions to to see if they are correct.

"Evidence" is anything that should cause you to update your beliefs. The simulations show that a certain possible phenomenon is consistent with known physics when this was not known before, and therefore they raise the likelihood that the phenomenon exists.

> "Evidence" is anything that should cause you to update your beliefs.

No, this is too broad. We are talking about testing scientific theories, not just updating a random person's beliefs. "Evidence" is what you compare theoretical predictions with to test theories.

> The simulations show that a certain possible phenomenon is consistent with known physics when this was not known before

More or less, yes.

> and therefore they raise the likelihood that the phenomenon exists.

No, this is a non sequitur. Knowing that a phenomenon is consistent with the laws of physics tells you nothing about whether that phenomenon actually exists. The set of phenomena that actually exist is too tiny compared to the set of phenomena that are consistent with the laws of physics for knowing something is a member of the latter to give any useful information about the former.

Your argument ultimately comes down to an argument from authority about the use of the word "evidence". Arguments about word definitions are uninteresting. What is substantively at stake is if a simulation like the one in the paper can provide information about a scientific phenomenon. Because a pure simulation can cause a change in beliefs, it does provide information.

> Your argument ultimately comes down to an argument from authority about the use of the word "evidence".

No, it comes down to an important substantive point about what the word "evidence" implies and that that implication is not valid. See below.

> What is substantively at stake is if a simulation like the one in the paper can provide information about a scientific phenomenon.

And the answer to that is obvious: a simulation can provide information about what a particular scientific theory predicts. It cannot provide information about what the real world actually does. Only the real world can do that. But using the word "evidence" to describe the output of a computer simulation obfuscates that vital distinction. That is why I objected to it.

For a real-world example of why obfuscating this distinction is not a good idea, see kadonoishi's post downthread.

> a simulation can provide information about what a particular scientific theory predicts. It cannot provide information about what the real world actually does.

If the simulation had come out in the negative, and it turned out this phase of H20 was totally inconsistent with all known physics, that would cause any reasonable person to assign a lower probability to the proposition that this phenomenon exists in the world. That's providing information about the world. Since the outcome would provide information in the negative, it also does in the positive.

I agree that there's an important distinction to be drawn between observational evidence vs. outcomes of simulations in terms of how they provide information, but they both provide information that was not accessible before.

> Since the outcome would provide information in the negative, it also does in the positive.

No, this doesn't follow. The two cases are not symmetric. Ruling out a phenomenon (if we assume for the sake of argument that a "negative" simulation result can actually do this--in reality things are quite a bit more complex, the simulation was not run as a binary yes/no test of a "phenomenon") rules it out. But showing that a phenomenon is possible doesn't tell you anything useful about whether it actually happens.

I think you're probably right that it's better to call this a prediction rather than evidence, but there's no consensus on this. Right now I don't think it's reasonable to say this article is making a mistake using the term in this way. It's not wrong. Maybe it should be? Sure.

> Right now I don't think it's reasonable to say this article is making a mistake using the term in this way.

I disagree, because I think the ordinary lay person takes "evidence" to mean what I said and would agree that a computer simulation is not evidence. And this article is supposed to be for the ordinary lay person.

If HN can't agree, I see no reason to suppose all lay people would unite as one.

My problem with calling this evidence is that the simulation itself is taken as the theory being tested, without saying so. Nothing wrong with that, but it's an extremely specific theory. So specific, we can already call it wrong.

It's evidence if it can be used to falsify theories, imo. That is true of some computational science, i.e. if you use a computer to make predictions from a theory to compare the predictions with observations.

Exactly, if you can falsify theories subject to the assumptions of the simulation, that’s still evidence.

> if you can falsify theories subject to the assumptions of the simulation, that’s still evidence.

No, the falsification itself is not evidence; it's a falsification. Evidence is what you compare the predictions of the simulation to, and then you call the theory falsified if the two don't match.

I spent some years doing computer simulations of cancer screening. The confusion between the output of the simulation, which is not evidence, and real evidence, led me into a nightmare of the Hall of Mirrors, where everything I attempted to learn about cancer screening turned out to be merely a reflection of my own assumptions. Everywhere I turned it was another funhouse mirror, where the computer warped my own ideas because the code I wrote was not a perfect flat reflection of what I meant, because my code introduced distortions, which took immense rigor to avoid.

The confusion between real evidence, and the output of my computer simulation which was not evidence but I _confused_ it with evidence, led me down a dark path lost in a labyrinth of funhouse mirrors of my own construction.

Listen to pdonis, people.

There is already a word for what you are describing, and that's proof. Evidence is a far broader term. It can be weak, strong, misleading or just indicative.

If I have a problem with the title it's the word "shows", this evidence indicates a behaviour water may have, it doesn't show that it has it.

> It's evidence if it can be used to falsify theories

No, this is too broad, because falsifying theories takes two elements: a prediction, and the evidence that the prediction gets compared to. Only the second is evidence; the first is not. And simulations only give you the first.

Reply on news.ycombinator.com