Hacker News Reader

Comment by heyodai | Hacker News Reader

Medieval astronomers were able to predict the orbits of planets surprisingly accurately, within 1/10th of a degree, even though they were assuming the Earth was the center of the solar system. They invented a very complicated system of deferents and epicycles (circles within circles). It was completely wrong but the outputs were surprisingly close to reality.

If we were using LLMs to analyze astronomy, we might just get increasingly complicated epicycles and never realize that the Sun is the real center of the solar system. We then never learn about the anomalies in Mercury's orbit that led to the theory of relativity.

So I agree. I seriously question how valuable LLMs can be in science/math beyond working as advanced search functions.

View on HN

The epicycles model represents one arbitrary periodic function as the sum of multiple well understood periodic functions (uniform circular motion, i.e. complex exponentials). This is the same idea we use in a more systematic way today by way of the Fourier series, etc.

It's quite a good method for approximating any arbitrary smooth periodic function.

My best guess is that the originators of this model (perhaps as early as Hipparchus, but at any rate someone by the time of Ptolemy) didn't think there were literally circles being combined, but had a pretty clear idea that they were trying to make an approximate model to fit the data. A previous model, from Eudoxus, was also probably pretty accurate but more cumbersome to compute with, involving approximation of the visible planetary motion by the combination of uniform rotations of some imagined sphere(s) centered on the earth. (We don't know exactly because the books about it don't survive, so all we have are vague descriptions by other people.)

It's almost like the remarkable fact that many things turn out to be turing complete systems that you might not expect (that is, able to perform any operation that any other computer can perform, albiet more slowly). Board games, Chomsky's generative grammar, bacteria (in a sense). Natural languages, programming languages, odd combinations of logical operators.

Any number of things can serve as a basic vocabulary to postulate entities, and do so in a textured way that matches some parts of underlying physical reality.

It's tempting to look at this and make the tragic jump into anti-realism because of the interchangeability of so many forms of representation. But I think as long as you have a combination of pragmatic attitude, treat knowledge claims as provisional and converging on truth, there's a cash value to descriptions that makes them legitimately truth tracking regardless of how they're formed.

>If we were using LLMs to analyze astronomy, we might just get increasingly complicated epicycles and never realize that the Sun is the real center of the solar system

I think this is a fantastic point and this kind of complexifying can and will happen. And I think future of online debating and propaganda is going to complexify in a similar way.

I will say though, I think that can be controlled by building in principles that prefer competing theories on the grounds of cogency, and that cancel out competing theories by reducing them to the parts between them that are equivalent.

My hard disagree will be with this: I don't think at all that we would "never" have got to the theory of relativity. It reminds me a bit of the "embodied cognition" argument against simulated brains. That argument suggests you can't "just" simulate brains, because actual brains are the totality of their embodiment in bodies and environments. Regardless of whether you agree with that line of thinking (I don't), we can assume it's true, and change the target of our simulation to brains + bodies + environments. The problem is bigger, but still perfectly amenable to the same methods.

I think the same is true of theorizing about science and imposing metatheoretical constraints. I think you can optimize for healthy hypothesis production just like you can for first-order reasoning about data. It's definitely harder but not necessarily a difference in kind.

The more I read about LlMs the more I wonder - there’s maybe two levels of emergence. LLMs obviously do more than the underlying vectors suggest. But it seems like they could have a hard limit of their training data. Filling in gaps in our knowledge to be sure, but always feeling like maybe they’re just about to hit the frontier.

If Epicycles made good predictions, then they were not "completely wrong". Nor is putting the sun at the center "right" so long as the predictions are still good. It is a matter of taste, not correctness (it is, however, very good taste).

I think you might be underestimating how transformative an "advance search function" could be. If the search engine knows how to design and carry out experiments in order to synthesize new evidence that allows it to ensure that the answers it provides are well supported, then humans are essentially out of the "truth" part of science, leaving them only to advice on the "beauty" parts, which might be ok, but it's a pretty big shift.

sorry, saying that "it's a matter of taste" whether the earth revolves around the sun is completely ridiculous. making useful predictions is one thing, but actual truth is not a "matter of taste".

Epicycles weren't untrue, they're just an unnecessarily complex model that can be used to describe what picking a different point of reference can yield as a much simpler model. Not all heliocentric models got everything right just because they picked a simpler reference point for producing a model. Judge a model based on its accuracy, not its point of reference.

> Epicycles weren't untrue, they're just an unnecessarily complex model that can be used to describe what picking a different point of reference can yield as a much simpler model.

That’s another way of saying it’s untrue.

> Judge a model based on its accuracy, not its point of reference.

Something can be both accurate and wrong, in the manner of a model that pinpoints the cause of death from cancer as admission to a cancer ward.

That's an interesting stance to take. So if you have multiple theories all of which make the same predictions, are you saying that the simplest one is the true one?

Are you sure there will always be just one simplest theory? Or are you using some other criteria to pick it?

No, the one that is actually explaining the phenomenon is the true one. The concept I'm conveying is sometimes called "mechanism of action"

There's a basic principle called Ockham's Razor that I'm sure you're aware of, which proposes that the simplest theory is the most likely candidate, but that's a heuristic, not a rule.

Another relevant concept here is that the map is not the territory. All theories are attempts to express some kind of underlying truth via some sort of formal system. Some of those theories map much more closely to the truth than others, in a way that's independent of accurately predicting observed behavior to date.

The philosophical problem of that approach is that it's impossible to assess whether you've arrived to the underlying truth, as you have no way to perceive it directly.

All you have is the different theories, so there's no definitive way to assess their closeness to truth other than by their predictive accuracy. If you don't have the physical capacity to leave the room and observe the world, all you've got is the maps themselves and how well they predict travel times and features found along the way.

I question whether a unique mechanism of action exists for all phenomena, but let's suppose that it does. Can you always know which one it is?

The only tool you have is the inductive strength of a big pile of observations in which correlations emerge. You can do experiments to prune certain causal stories, but there's never time to eliminate all possible stories. You end up with something that makes sense for you, in terms of the equipment that you have, and the theories that you and your peers use, and it's good enough, so we move forward with it.

I can see how it might be useful to document a particular explanation as "the" explanation, especially if you're communicating why a drug works because it will lead to useful conclusions about who should or shouldn't take the drug.

But when it comes to speculation about whether a given astronomical body will be visible at a particular time, I think science would have progressed much more quickly if we had not held so tightly to the notion that one explanation is intrinsically correct, and we're probably holding ourselves back now in other domains by doing the same.

It's not illegitimate to model the solar system from the perspective of earth. It just makes the model more complicated than if you do it from the sun's perspective.

I think we disagree pretty fundamentally on what something "being true" means.

It's possible to come up with an "unnecessarily complex model" for why someone accused of a crime did the crime, but that has completely no bearing on whether the defendant is actually guilty. If the defendant is, in fact, innocent, then the prosecution's model is, in fact, untrue - no matter how well it fits the evidence.

> no matter how well it fits the evidence

Are you proposing that an explanation can be true even though there's a different explanation which better fits the evidence?

Or are we talking about cases where both candidate explanations fit the evidence equally, and you have access to some kind of intuition which tells you which of the two are true?

> Are you proposing that an explanation can be true even though there's a different explanation which better fits the evidence?

Such a proposition would be correct; compare the concept (and name!) of "overfitting" in statistics.

I'm talking about a case where the _best explanation_ you have suggests the defendant is guilty, however the _actual truth_ happens to be that they're innocent

> but actual truth

there is no "actual truth" - there is only what you can measure reliably

https://en.wikipedia.org/wiki/Epistemology

unless you subscribe to an entirely solipsistic worldview, "actual truth" obviously exists regardless of your ability to measure or perceive it.

Brother you're wrong and there's ample scholarship you can consult to correct your misunderstanding. The Wikipedia with sufficient citations is right there. Feel free to educate yourself.

I do not appreciate your condescension.

Either state your argument coherently so as to address my completely valid objection, or move on if you're not interested in having a discussion.

Arguing by sending bare links to wiki articles isn't productive and doesn't make you look particularly good.

Not defending GP, but your claim that actual truth "obviously exists" is equally condescending and empty of backing evidence.

Would you say the continuum hypothesis (or its negation) is "actual truth"? Or does the "actual truth" become something closer to "actually, there are models of ZFC where CH holds as well as models where it doesn't hold, or maybe ZFC is already inconsistent; we don't really know, nor can we."?

Similarly, on the subject of astronomical reference frames, I'd argue the "actual truth" is closer to this:

> Any reference frame that locally obeys Newton's law of inertia is equally valid. Some are more useful than others depending on the problem you are studying. The notion of an "absolute center" is purely semantic, and a meaningless human construct, until one places bounds on the system under study, wherein the center of mass becomes the most reasonable choice at sub-cosmological scales.

> "actual truth" obviously exists regardless of your ability to measure or perceive it.

a reasonable person would at least peruse the literature before making such strong claims as

> Either state your argument coherently so as to address my completely valid objection, or move on if you're not interested in having a discussion.

i'm not interested in educating you on thousands (literally dating back to plato) of years of scholarship/philosophical discourse on the topic of epistemology. that's not my responsibility because you are not paying me for this - typically people charge large amounts of money for this kind of work (uni professors are well compensated).

lucky for you others have done lots of work making this information available to you for the low low price of "reading". that's why i provided the link.

Sorry, you're the one making ridiculously strong claims like "actual truth doesn't exist". The onus is on you to prove them.

Like I said in a different thread: thousands of years from now, it might literally not be possible to prove that you ever existed, but that wouldn't change the "actual truth" - the fact that you did exist, and do things. Stuff happens outside of your person, in the real world, without regard for our ability to perceive/articulate/"know" it. It's only possible to say things like "actual truth doesn't exist" if you think your mind is the only thing that really exists.

> i'm not interested in educating you on thousands (literally dating back to plato) of years of scholarship/philosophical discourse on the topic of epistemology.

Cool. Then don't start a discussion with me.

> thousands of years from now, it might literally not be possible to prove that you ever existed, but that wouldn't change the "actual truth" - the fact that you did exist, and do things.

My man forget thousands of years in the future you don't know whether I exist today lololol

> Cool. Then don't start a discussion with m

I didn't? I told you were wrong and that's it (and then you got pissy). I know that's what happened because each of ours responses are recorded on this indelible/irrefutable medium lololol

Is it useful to invoke it though? Once two people claim to know the actual truth, yet they disagree, you're at an impasse. There's no method for resolving which allegedly obvious claim is in fact the truth.

Predictive power on the other hand, gives you a framework to decide which story is more useful.

I mean to be completely accurate the Earth doesn't strictly orbit the Sun. Both the Sun and Earth orbit each other around a point at their "combined" center of mass, the barycenter - https://en.wikipedia.org/wiki/Barycenter_(astronomy). This is something you need to take into account for all the planets in the solar system and as a result, the "center" of the whole system is constantly moving. The solar system's barycenter is often not even inside the Sun.

I think that sort of proves the point that the OP was making, which is that you can create a functional model around any arbitrary location and it can be valid (as long as valid means: it can make useful predictions). When you put the center of the Sun at the center of the model, you still need your models to account for the "orbital wobble" happening as a result of the Sun doing a little dance with Jupiter (and all the other masses that orbit it) in the equations for the determining the relative positions of the Sun and Earth.

Using the Sun as the center versus the solar systems barycenter are both somewhat "arbitrary" in that way, and depend on what you're trying to do with the information - how accurate you need it to be, and how complicated you're willing to make the calculations to hit higher accuracies.

If I just add a few more parameters to my model I can make it fit anything!

Sure, it depends on what you're trying to do. Fit an equation or understand it. Yes, you can use a reduced mass to simplify the equations and make it more accurate, which usually isn't necessary when the new center is at less than 1% of the radius of the Sun (note ellipses have 2 foci from Kepler/Newton). Even Jupiter's center of rotation is at the surface of the Sun, and both have eccentricities <1%.

Of course, if you're using epicycles, you're never going to get Einstein's elliptical precession correction to the orbit of Mercury. Simpler, is usually better for models, even when you need greater and greater accuracy, overfitting is the enemy of explanation (to coin a phrase).

All models are wrong, some are useful. - Box

Agreed. So to return to the root comment... if AI can generate for us all of those "wrong" models in a consistent way, then the job that remains for humans is to decide which one or ones would be useful. I bet it turns out to be a lot more work than it sounds like.

> If I just add a few more parameters to my model I can make it fit anything!

Dealing with people wanting to do this is story of my current professional life lol I would love to get them onto the same page that chasing down a final <1% improvement isn't (always) worth it

"All models are wrong, some are useful" - this is what I was trying to say, I was just being a bit snarky/pedantic because the commenter above seemed to be trying to pull a "gotcha" saying that the "truth" of something overrides the utility of it.

My point is that the laws of physics work just fine in a wide variety of mathematical framings. It's sensible to pick a framing that keeps the complexity of the equations as low as possible, but you're not incorrect if you pick a frame that puts the moon in the center or any other point. The accuracy of the theory's predictions are completely separate from where it applies labels like "center".

  > you're not incorrect if you pick a frame that puts the moon in the center or any other point

I think you're conflating different things. Yes, we can predict motion from the view of any point and that would likely involve epicycles. But that's not what people were after. Causality matters, a lot. It's kinda the whole point of physics (and what makes it so expressive and powerful).

What classical relativity does not allow you to do is place yourself at an arbitrary point and call that the center of mass of the system (i.e. things orbit around you). That's not invariant. It's a very different thing than predicting the motion of objects from your vantage point.

100% agreed, but Ptolemy and Copernicus weren't talking about mass or causality, they were just looking for somewhere to put the origin, and making predictions without reference to gravity. Picking an origin such that it happens to align with a barycenter would be in good taste for an audience with our modern understanding of gravity, but picking one elsewhere would not be incorrect so long as the predictions held up.

And that's the part that I'm suggesting AI might be unable to solve. Even if it can effortlessly generate theories that are consistent with both evidence and specification, there's still a lot of thinking to do about which ones we should specify for use in different situations.

Even if a "Theory of Everything" can exist, it may not be suitable for all situations for squishy human reasons besides anything having to do with correctness.

  > but Ptolemy and Copernicus weren't talking about mass or causality

Except they were

My critique is that there's a difference between "x is the center" and "motion looks like y when sitting at x".

Those are different claims with very different consequences. You're dismissing this as minutia but that minutia is the critical point. In a geocentric model you need to explain why things revolve around the earth. In a heliocentric model you need to explain why things move around the sun. Predicting the motion of the planets doesn't answer these. But answering these *correctly* allows you to build a generalized model in which you can predict the motion of planets, in any arbitrary system.

I'll also add that I love the geocentric model as a good illustration of why observation alone can't give you causality. There's far more to science than data fitting. In fact, that's why all the mathematicians are getting upset and what Tao has been talking about this whole time

I don't think Copernicus did any explaining why at all. That took another 150 years.

We now bind our physics to our astronomy, but they didn't then. That's what I mean by it being a matter of taste, not of truth.

If you were prompting a godlike LLM for a theory of the heavens in the 1540's you probably wouldn't have used words like "mass" at all, and the theory that it spit out would be similarly absent that concept.

They both revolve around their barycenter. So neither one revolving around the other is ground truth. In practice either one can be the right choice (earth at center for video game, sun at center for astronomy predictions).

Galactic core at center for other astronomy predictions...

And where's the barycenter located? When keeping an open mind make sure your brain doesn't fall out.

People are using natural language. And considering the barycenter sits inside the sun or close to it (relatively), it isn't that wrong of a statement.

While you're technically correct, everyone already understood. We could be more technically correct by talking about the galactic barycenter, or even more by talking about the universe as a whole, but the accuracy gains by that added complexity doesn't benefit the conversation

GR says that any falling (or orbiting) reference frame is equally valid, locally obeying Newton's laws. So in a precise sense, geocentric and heliocentric reference frames are equally valid. Both can claim to be "actual truth". Of course, a heliocentric reference frame is a lot more useful for studying planetary orbits since it gives rise to simpler equations of state. But that doesn't make the geocentric frame "wrong".

Put differently, would you say a heliocentric reference frame is also "wrong" since the sun is actually orbiting around Sagittarius A*?

Taking your argument to its logical conclusion, one would conclude that the [comoving reference frane](https://en.wikipedia.org/wiki/Comoving_and_proper_distances) is the only "true" one. And at last there is real validity to this claim, since the CMB does pick out a preferred velocity at each point in spacetime. But it is pretty impractical to use comoving coordinates to study problems outside of cosmology.

Ultimately, it stops being a matter of "right" or "wrong" but rather one of picking the right tool for the job you're working on.

It might have been a matter of taste in the 17th century. The Church was willing to accept a model where the sun goes around the earth, and the planets around the sun. This model agreed with observations, and was in good taste as well, as it avoided the problem of why it doesn't feel like we're in motion, and why we don't get flung off the earth.

Then Newton came along and blew all of the models out of the water. Nobody's at the center of the universe [0]. A single theory of gravity models both planetary motion and why we don't get flung off the earth. And we can "feel" the rotation of the earth by experiments which came along later such as Foucault's pendulum and the Coriolis force.

[0] A puzzle I like to taunt my friends with: The earth is in fact equidistant from the edges of the observable universe in all directions, so we must be in the center after all. ;-)

IIRC the universe is a pretty non-unifrom shape.

I think "there is no center" and "where you put the center is a matter of taste" are more or less equivalent. And if you're confronted with a pencil and paper and asked to draw the solar system, the latter is more useful.

More useful to us because we know more about how it works. This is why I was specific about the time period.

Epicycles just had a poor predicting power; they required many more free parameters to describe reality, compared with the heliocentric model.

Knowledge can be framed as information compression (this actually applies to LLMs as well, amusingly). The heliocentric model plus Newton's gravity are an amazing compression of information related to dynamics of celestial bodies, which then enables related predictions that would be much more difficult to arrive at if you start from epicycles.

Is amazing compression of information a goal of the universe or of humans attempting to understand the universe?

Also, most models are lossy compression. The heliocentric model is amazing because it is wrong. Complications are simply ignored.

The introduction of LLM in mathematics is no different than the introduction of automatic calculators. Probably the calculators (computers) where even more groundbreaking if you think so (something that was impossible to calculate or simulate it was now possible).

Just don't believe at all the shitty marketing that AI companies are creating to make you believe that they have found the Holy Grail

I don't think checking our work is a uniquely human trait. It's our uniquely human traits that made us willfully ignore the better model in favor of increasingly complicated epicycles.

> If we were using [humans] to analyze astronomy, we might just get increasingly complicated epicycles and never realize that the Sun is the real center of the solar system.

Current models have undergone massive amounts of RL to "get shit done." There's incentive toward reasoning, but probably little toward elegance. Every engineer is familiar with the impulse to just hack things together.

But this need not be a permanent state of affairs. The model labs are probably already examining the elegance problem, since it is such a common complaint of researchers reading these machine generated proofs and papers. And I haven't seen anything disproving the notion that elegance can be RL'd.

Precession of perihelion of Mercury was not the motivation of the discovery of general relativity theory. It's only an evidence that convinced why the theory ought be true.

What you have to believe is that somehow the entire field of math was just leaving open problems on the ground that were actually easily solvable and were essentially free Fields Prizes, tickets to a lifetime of Fame and Academic superstardom. Orr LLMs are actually doing something novel.

Maybe some of them are like finding the 1 millionth digit of pi. Not hard, but impossible without the right technological advancement.

Now that we have LLMs, those that are hard for people but easy for LLMs will get cleared out, leaving those that are still hard for both.

>Maybe some of them are like finding the 1 millionth digit of pi.

Yes some of them were clearly like that but there are probably 5-10 which many mathematicians have said were massive results and would all but guarantee a Fields Medal to any human that had solved them. Navier Stokes, Quasi Riemann, etc.

Of course. But we might have found out the sun is the center of the solar system much sooner. Who knows!

I feel like there's a lot of "humans are special" in this thread. There's a good chance we're not.

> It was completely wrong but the outputs were surprisingly close to reality.

that's not how physics works. epicycles weren't wrong. there is no wrong/right - modern scientists don't believe in platonism. physics is a collection of models which are empirically tested. if epicycles makes predictions to more decimal places than relativity then epicycles are "right" and relativity is "wrong".

now think about how this translates to LLMs...

> physics is a collection of models which are empirically tested. if epicycles makes predictions to more decimal places than relativity then epicycles are "right" and relativity is "wrong".

Most models have relation to each other so it does not suffice to take one in isolation. It's one continuous world (in our knowledge) so almost everything relates to one other. The truthiness of something is not whether it fits some particular phenomena, but how does it generalize.

> Most models have relation to each other so it does not suffice to take one in isolation

You don't know what you're talking about (most people on here do not).

https://en.wikipedia.org/wiki/Ansatz

> You don't know what you're talking about (most people on here do not).

> https://en.wikipedia.org/wiki/Ansatz

  After an ansatz, which constitutes nothing more than an assumption, has been established, the equations are solved more precisely for the general function of interest, which then constitutes a confirmation of the assumption. In essence, an ansatz makes assumptions about the form of the solution to a problem so as to make the solution easier to find

I don't see where this contradict my statement. A good answer does not means the formula used is correct.

> A good answer does not means the formula used is correct.

yes it does. that's exactly what it means.

Reply on news.ycombinator.com