Hacker News Reader

Comment by blueone | Hacker News Reader

> what basis do you have for assuming an LLM is fundamentally incapable of doing this?

because I have no basis for assuming an LLM is fundamentally capable of doing this.

Good on you for spelling out this reasoning, but it is manifestly unsound. For a wide variety of values of X, people a few years ago had no reason to expect that LLMs would be capable of X. Yet here we are.

In 1989, Gary Kasparov said that it was "ridiculous!" to suggest a computer would ever beat him at chess.

"Never shall I be beaten by a machine!”

In 1997 he lost to Deep Blue.

Yeah, and back then people moved the goal posts too, saying Deep Blue was just "brute-forcing" chess (which isn't even true since it's not a pure minimax search).

Deep Blue was brute forcing chess in the sense that AlphaGo wasn't brute forcing Go.

Both of them contained a search algorithm that explored some moves from each considered position, usually not all moves. Both of them contained logic (learned or programmed) to evaluate moves and/or positions.

The differences between them are many, but brute force doesn't enter into it in either case.

And today he's got salient observations on politics which hold much of his attention, and Deep Blue is shut off and has done nothing further.

Not a good argument for turning everything over to the Deep Blues. What's Deep Blue done for me lately?

This is something that could be demonstrated rather than just argued.

Train an LLM only on texts dated prior to Newton and see if it can create calculus, derrive the equations of motion, etc.

If you ask it about the nature of light and it directs you to do experiments with a prism I'd say we're really getting somewhere.

We tried this experiment with humans, back in the 17th century, and only a few[1] out of millions managed it given a whole human lifetime each.

[1] Obviously Newton counts as one. Leibniz like Newton figured out calculus. Other people did important work in dynamics though no one else's was as impressive as Newton's. But the vast majority of human-level intelligences trained on texts prior to Newton did not create calculus or derive the equations of motion or come close to doing either of those things.

Newton did it at 23 and there would have been very few people with mathematical training. The LLM would be trained on the entirety of recorded human knowledge and mathematics up to that point, and would get to use a lot more energy so it still has a massive material advantage over young Isaac. Yet I don't believe calculus would magically appear in its response.

A good way to look at it is to compare it to today: LLMs are already trained and are operationalizing a lot more mathematical knowledge than any human, including experts.

Why are they not coming up with paradigm shift in knowledge expression/discovery like humans did back then?

Are we just not prompting them right?

LLMs have been trained on a lot more data than any single human (text wise at least) for years now and these sort of results have only been possible for the latest crop of models in the past few months. Models get better as they get better.

The argument is whether models of today, suitably trained on pre-17th century data (if comparable quantity was available) would be able to "invent" calculus et cetera.

If we believe today's models are sufficiently capable to have been able to do so, why are we not getting these types of results today compared to the entire world knowledge and especially math?

Are research mathematicians simply not prompting LLMs in the right way?

Except this has been said since the 2010's and has been proven wrong again and again. Clearly the theory that LLM's can't "extrapolate" is woefully incomplete at best (and most likely simply incorrect). Before the rise of ChatGPT, the onus was on the labs to show it was plausible. At this point, I think the more epistemologically honest position is to put the burden back on the naysayers. At the least, they need to admit they were wrong and give a satisfactory explanation why their conceptual model was unable to account for the tremendous success of LLM's and why their model is still correct going forward. Realistically, progress on the "anti-LLM" side requires a more nuanced conceptual model to be developed carefully outlining and demonstrating the fundamental deficiencies of LLMs (not just deficiencies in current LLMs, but a theory of why further advancements can't solve the deficiencies).

Incidentally, similar conversations were had about ML writ large vs. classical statistics/methods, and now they've more or less completely died down since it's clear who won (I'm not saying classical methods are useless, but rather that it's obvious the naysayers were wrong). I anticipate the same trajectory here. The main difference is that because of the nature of the domain, everyone has an opinion on LLM's while the ML vs. statistics battle was mostly confined within technical/academic spaces.

> Clearly the theory that LLM's can't "extrapolate" is woefully incomplete at best (and most likely simply incorrect).

What example is there where an LLM has extrapolated? All I've seen is a data set so large and an extra decomposition process making it so interpolation feels like extrapolation if you don't look close enough.

> but a theory of why further advancements can't solve the deficiencies

How about LeCun's?

Reply on news.ycombinator.com