No physics? No problem. AI weather forecasting is already making strides

Thank you!

That's what model interpretability research is. You can train an interpretable model from the uninterpretable teacher, you can look at layer activations and how they correspond to certain features, or apply a hundred other domain-specific methods depending on your architecture. [0]

Sadly, insight is always lost. In a noisy world where even with the best regularization, some fitting on it, or higher order features that describe it, is inevitable for maximizing prediction accuracy, especially if you don't have the right tools to model it (like transformers adapting to lacking registers [1]) and yet a lot of parameters within chosen architecture.

What's worse, bad expectations are often much worse than none. If your loan had been denied by a fully opaque black box, you may be offered recourse to get an actual human on the case. If they've trained an interpretable student [2], either by intentional manipulation or by pure luck, it may have obscured the effect of some meta-feature likely corresponding to something like race, thus whitewashing the stochastically racist black box. [3]

[0] "Interpretability in ML: A Broad Overview" https://www.lesswrong.com/posts/57fTWCpsAyjeAimTp/interpreta... [1] "Thread: Circuits" https://distill.pub/2020/circuits/ [2] "Why Should I Trust You?": Explaining the Predictions of Any Classifier" https://arxiv.org/abs/1602.04938 [3] "Fairwashing: the risk of rationalization" https://proceedings.mlr.press/v97/aivodji19a

I think we understand the processes just fine. The issue is that we can't get a "closed form" solution for 10^100 interactions per planck time.

I think at some point it is worth admitting that there are variables you can't account for. Like the precise geography - the models model an area 1 mile square as a single vector, maybe even more coarse. They don't model every tree, rock, and bush. In a neural net you can just have "weight goop" which accounts for the net effect of these unmodeled features, but in a traditional model adding "fudge factors" and extrapolating back from the model to points of interest is tricky.

> and therefore does not really provide understanding of the underlying process

What is “the underlying process”?

For example, Newton was able to model gravity quite successfully without ever being able to “understand the underlying process”. In fact, physics today still doesn’t have a good grasp on what gravity is. Yet we use the models and equations all the time

In a way, physics is also a collection of black boxes, perhaps just seemingly more elegant boxes

Wouldn't you be able to somehow couple it with another model that takes the NN data and somehow untangles it's convoluted logic into an isomorphic human readable equation, ie. a model that has one task and that is translating NN logic into human equations.

The training data could be real physics in a simulator held up against evolutionary driven AI logic that competes against it with various goals that are then evaluated and if given a high score then marked as isomorphic and given enough runs you'd get a dataset.

My comment was in the context of the article, which states that AI doesn’t need physics

Having models doesn’t imply what you probably mean by “knowing about physics”, it just means having a representation of something that is not that something (like a map of the world is a map, not the world)

So, depending on how the dog’s understanding of the world works, it maybe doesn’t need any models, I can’t really know

But I do know, that if I want to describe anything, using any symbols whatsoever, then I’m implicitly creating a model of what I’m trying to describe with the symbols

So, if I’m trying to communicate and understand physical phenomena using AI, then I’m implicitly creating a physics model, whether I want to call it that or not

A charitable interpretation of nico is that he was saying a well-trained NN is itself a model of the world. If it can tell you what a system will do given some inputs, then it functions as a model. While internally it isn't creating a model that we could understand, it does "model the world" in the sense that we can treat it as a model

I’m saying the person setting up the NN is doing so by following some model of what outputs to expect from the inputs

To train a NN you need a training set of data, that data follows a certain order or pattern that represents the model that the person has

As long as the NN behaves as expected by the person setting it up, then it is useful as a model

How useful? That depends on what you are trying to do, your model and the data

If you want to take it to its extreme, all language is a model. Even what I’m expressing right now with this text. It’s just a model of what I’m thinking and the meanings I’m trying to convey

Observation assimilation is a huge field in and of itself. Observables have biases that have to be included in assimilation, they also have finite resolution and so observation operators need to be taken into account.

Satellites also measure the temperature/height of clouds, and there's also some data from aircraft. A lot of commercial aircraft automatically report the temperature/pressure as they fly. The only problem is that a lot of their flight is in the stratosphere, but they give good data in their climb/descent.

Because we have well-measured inputs for CO2 between basically 250 PPM and $current_day PPM. Do you see how this statement

> Eh, atmospheric CO2 (and other GHGs) is just another input parameter

only works for CO2 inputs outside our measurements, if the climate response to those inputs is linear (and thus predictable from the responses we have already seen)?

I claim that the climate response to CO2 forcing is, in fact, strongly nonlinear, and further that it's nonlinear for other "unusual inputs" - not just CO2 - things like sea surface temperature or unusually low pressure troughs. So-called extreme weather. I can't bring up good citations at the moment, sorry, but here's a somewhat exaggerated thought experiment:

Take an AI model trained on weather measurements ~1970-2024. Also take a model of the primitive fluid equations on a rotating sphere. What predictions might you expect from each one for an asteroid hitting an empty patch of the West Pacific?

But the balloons stay up for weeks and float around; doesn't that cause problems?