Comment by fwlr | Hacker News Reader

Comment by fwlr | Hacker News Reader

Mathematics is like this. You read the first symbol in the paper, it is a wiggly triangle, what does that mean? Well you will find out that symbol means the Constant or Operator or Set or Function belonging to So-and-So with the unfortunate name. Well now you know it is called Grossediche’s Member, what does that mean? You will find out it is defined in these dozen lines in Grossediche’s seminal paper, you will need to read the entire paper to make sense of these dozen lines, you will need to read everything he published in this particular decade to make sense of the paper. Each of the dozen lines is jam packed with other symbols, for each of those you will have to repeat this entire process, with another stack of papers, from another unfortunately named mathematician. Now you have a firm grasp on Grossediche’s Member, you return to the original paper. You read the second symbol in the paper, it is a half-melted letter t, what does that mean? Well, …

Behind each symbol is a whole paper, behind each paper is a whole life’s work, and so on. With this in mind, it is perhaps not so surprising that language models operating on embeddings are extraordinarily well-suited to this particular task.

That is research math.

If you read an introductory book (like Algebra: Chapter 0 by Aluffi, yeah the choice is a bit naughty), it doesn't assume (too many) prerequisites.

And after you've read enough of these (e.g. when you have a BSc in math), research papers are more accessible.

LLMs don't understand things as human mathematicians do, even though they are very good at finding analogies and similarities. Their advantage is a larger search space (experience) and search speed, not better understanding.

> LLMs don't understand things as human mathematicians do

You say it like it's a bad thing!

I gestured at it with “embeddings” but to spell it out, I’m more or less claiming that the only sense in which LLMs “understand” a given thing is as big list of all the things it is related to (implemented in the form of a vector embedding). And all those things it is related to, each one of those things is just a big list of yet more things it is related to, and so on. I do not mean that it is eventually hitting a “base case” that contains semantic meaning and then transforming it according to the relationship path it took to get there; rather, I mean that it follows enough n-th order relationship links that the shape of the relationship to the future thing (i.e. what it is generating) is constrained enough to pick output tokens on the basis of their relationship to the current token alone.

As an analogy: if I give you a stream of numbers and you notice that the delta between number n and number n+1 is always 2, you now know enough about the relationships between the numbers in the stream to pick the next number without ever knowing what the numbers were.

Reply on news.ycombinator.com