Hacker News Reader

> No, it’s not.

What's "not"? His statement about the gravitational constant is exactly correct: we have no theory that tells us what the value of this constant should be, we have to get its value from measurements, and doing that is a process of statistical inference.

It is also true that the form of the theory itself--its equations--is not given to us magically but has to be developed by comparing predictions with experimental data, i.e., by a process of statistical inference.

Physicists don't usually describe these processes as "model training", but that doesn't mean such a description is wrong.

I guess one difference between the two is explainability. As we predict, test, and update, there is an ongoing narrative along the way of why we are making this change or the other. We can explain why we thought some scientific model was the correct one, even if it turns out to be wrong and needs to be udpated. In model training as it's used here, I don't think there is any such a thing.

That's really the divergence Norvig points to between Bregman's two cultures.

And there are plenty of cases where we have had essentially purely phenomenological models in physics (I mentioned this in another thread, but Landau theory) which only _later_ were systematized, so it's a set of processes on a continuum, not two binary opposites.

People _treat_ physical laws as absolutely true, because with high confidence they are and it's intellectually convenient, but they're really just models. We build models on those models, and mental models using our _interpretation_ of those models; no-one is denying the importance of model interpretability in all of this – but it's absolutely a tower of model-building, with more-or-less convincing explanations for the model parameters at various points.

Physics, or at least some domains of physics, are tractable using what one might term the "axiomatic style" because the models we have are a) staggeringly explainable and b) extremely robustly supported by measurement. Even that statement exists on a continuum: MOND is pretty damn phenomenological, for one example, and while we have pretty good quantum chemistry methods in the small, we certainly don't have practical ways of using those ab-initio methods for even mesoscale problems in solid state physics. Does that make mesoscale physics "not science"?

A science which rejects all phenomenology and which insists on building everything up from first principles isn't a science; it's mathematics. That's totally fine! It's just a different—related, but distinct—domain of study.

> there are plenty of cases where we have had essentially purely phenomenological models in physics

Actually, there is a fairly common point of view (which often goes by the name of "effective field theory") according to which all of our physical theories, even the ones we usually refer to as "fundamental" like General Relativity and the Standard Model of particle physics, are phenomenological; they aren't the actual "bottom layer" but something that emerges as an effective theory from other layers deeper down (which we don't have a good theory of at this point).

I came to argue in favor of aj7's argument as a physicist. I think we are all missing the forest for the trees here. what elevates the gravitational law to it's status and differentiates it from a simple model, and what i think the point that aj7 is trying to make, and also the point that norvig is choosing not to focus on here, is not the gravitational constant G. Noone really cares about the value of the constant and how it is approximated in this part of our spacetime etc [edit: of course they do care if it is postive, zero, or negative for reasons consistent to what i am saying later]. The "juice" here is the inverse square relation to length. The form of the equation is what is the deep insight and what also provides for that insight to transfer to other systems. Knowing the nature of the differential equation is of uttermost importance because then we can reason about possible and impossible outcomes of it just from it's mathematical (and only) nature. If these models in the future can be built in ways that they can be reasoned about symbolically like these systems that we strive for in physics, i think they will be elevated to laws (or proper theories as aj7 says) rather than models. and noone will care if they have a billion parameters as long as they have a certain structure that we can work with. I hope this makes some sense to support the counter-norvig argument in this (and i personally respect both chomsky and norvig infinitely of course. i just think norvig's wording in this specific part of the article is indeed crude and shallow, maybe just to make his counterargument)

I disagree. It’s true that we are used to thinking of gravitation as a theory that was double checked with data, but it is also pretty clear that good physical arguments are strong priors. Then, with experimental data, we have a classic prior + data posterior update which does cast all of this as a statistical model.

That is not at all the way Newton thought, though.

Then all scientific theories are “models.” And we are just very advanced “computers.” Nope.

The first sentence is wrong. And the second sentence is why we play the game.

Is the use of those quoted words what you dislike? Would you have a different sense of it if we replaced those words with "foo" and "bar" so that we could use more nuanced concepts?

> all scientific theories are “models.”

That's correct. They are. As the saying goes, all models are wrong but some are useful.

> And we are just very advanced “computers.”

To a physicalist, yes, this is true.

I think you're coming at this from a dogmatic interpretation and it's making it hard for you to see the alternate perspective.

All scientific theories are models; that's generally considered true by nearly all scientists. What would a theory be if it's not a model?

> Then all scientific theories are “models.”

I may be missing the context of what you mean by model and somewhat agree with your original critique of Norvig's comment. However, scientific theories are nothing but models. For example, general relativity is not gravity but is instead a model of gravity. Quantum mechanics describes quantum behavior but it is not quantum behavior. It is a model of quantum behavior.

When physicists say "is", they mean "is modeled by" or "is a model of", whether they think so or not. None of the mathematics, or physics, or what have you is the actual thing. They are nothing but models of the actual thing that have varying degrees of accuracy and precision.

> It’s a little horrifying that Norvig doesn’t seem to understand these nuances.

yes. its quite incredible actually. It is one thing to sort of tribulate about brain / cognition domains that nobody knows much about using a pseudo-language of "learning" or "intelligence" (or even the original sin: "neural networks") and it is another level altogether to expose publicly how much one is subject to "I have got a nice little hammer and to me everything is a nail" pathologies

Conflating the statistical verification of physical laws / theories with the theories themselves reveals in much clearer terms how limited the conceptual model the AI crowd want to inflict on us. We do know about more about physical laws than about the brain and they don't come about like they say. Go back to school.

Newtonian gravitation is absolutely “we propose a model, we fit it, here is the result, here are the error bars on _g_”.

Einsteinian gravitation is a better model. It fits the data better, that’s how we know.

These are models, exactly in line with Breiman’s “first culture”, data modelling - the model has been designed according to intuition about the underlying mechanism.

You seem to be arguing that if a model is not derived from first principles it is not science. This is ahistorical nonsense. Science has always been full of phenomenological models which haven’t lent themselves to direct interpretation. That doesn’t make the model wrong, it makes it uninterpretable. Some of them become interpretable later, some are replaced by interpretable models, often they give key insight to underlying relationships which must hold which then guide the search for a mechanism, and nearly always they’re profoundly useful. Consider Lev Landau’s order parameter model.

“Physical law” is just a synonym for “extremely high-confidence interpretable model”.

> “Physical law” is just a synonym for “extremely high-confidence interpretable model”.

You are abusing the term "model" to the point of ridicule. What is the "model" in Newtonian gravity? An iconic relation to which you might (over)fit some data "AI style"? The associated concepts (action/reaction, momentum) that float somewhere around the "formula" and are absolutely necessary to understand and use it but are not reflected in the slightest in this quantitative expression? The drastically alternative ways you can re-express the formula, which neverthless all contain exactly the same physical content? The explanatory (modeling) potential of the concepts (what happens when I split a mass in two?) that suggests a more profound link to what is actually being described?

At a very basic level it is true that we verify the conceptual constructs that we call physical laws by deriving certain statistically testable relations. But to argue that these derivations is all there is to it is simply... wrong.

> What is the "model" in Newtonian gravity?

F = G * ((m1 * m2) / r^2), plus, yes, the concepts of distance and inertial mass.

G (and the bounds on the precision of our measurement of G) is derived by fitting from experiment. It's a regression model which happens to have R^2 extremely close to 1, and that's why we can treat it as (nearly-)always true.

It's absolutely a statement about observed behavior which we then _interpret_ (incompletely, but usefully, as Einstein showed) as an inverse square law. This is precisely a _model_ of behavior. That model can be derived from first principles or it can be purely phenomenological, and different models are useful for different intellectual tasks.

Once you have a statement which is nearly always true, you can ask _why_ it's nearly always true, and that's very useful, but "law" really _does_ just mean "statement to which we haven't found counterexamples yet".

But the model is just a model. Science is the process of building, interpreting and invalidating models, and different pieces of science live at different _points_ on this continuum. Large language models in linguistics live off to an extreme point on it, but even there, models have designed-in inductive biases (eg the attention mechanism in most LLMs) which reflect the modeller's hypotheses about the structure of the problem.

You seem, like Chomsky, to want science to be much more Platonic and profound than it actually is. That's your choice.

Science consists of theories, not just models. This newspeak definition that science is just models is a form of scientism. It's reductionism which is why Chomsky is against it, and to attempt to rebut essentially an accusation of reductionism as being Platonistic is a mistake.

You're confusing _scientific_ theories (rebuttable empirical statements formalized typically through mathematical models) and mathematical theories (statements of fact derived from an axiomatized system).

Chomsky wants linguistics to be mathematics.

For some reason you want to hammer all of science into some sort of helpless pre-science. To a more-or-less direct statistical fit of measurements to a non-descript, intrinsically meaningless mathematical formula. A procedure that somebody might apply to an observational set using a generic family of formulae even if they have absolutely no clue of what is going on.

We know it doesn't work this way once we actually start building a real model and move beyond what is really just exploratory data science. We'd be nowhere with physical science if the mental models we construct (which have nothing particularly numerical about them in the first instance) did not actually have a coherence that is both amazing and extremely productive.

This is not some Platonic drift or remotely philosophical. This is how physics is done. Even casual familiarity with the history of physics and reading the writings of key thinkers would point to this.

Heck, in quantum mechanics the relation between the physical/mathematical model people came up with the actual "measurement" is profoundly non-trivial and does not remotely fit your bizarre reductionist "programme".

Thank you.

You're just using different words to say the same thing. The word "theory" doesn't have the significance you are attributing to it.

Yeah, I feel similarly. It comes across as if Norvig just doesn’t understand the perspective. He’s translating it to his terms and making strawmen by doing so.