Arithmetic Without Numbers – How LLMs Do Math
alvaro-videla.com
alvaro-videla.com
Very often it involves spatial thinking. Vide one example there:
> Once I mentioned this phenomenon to Andy Gleason; he immediately responded that when he taught algebra courses, if he was discussing cyclic subgroups of a group, he had a mental image of group elements breaking into a formation organized into circular groups. He said that 'we' never would say anything like that to the students. His words made a vivid picture in my head, because it fit with how I thought about groups. I was reminded of my long struggle as a student, trying to attach meaning to 'group', rather than just a collection of symbols, words, definitions, theorems and proofs that I read in a textbook.
And obviously Terrence Tao is up there in the response.
https://www.youtube.com/watch?v=U719vQz-WFs
Leslie Lamport : "I am not smart. I have the gift of abstraction."
Real mathematics isn't about details. Its about concepts and abstractions and how we compose them (LLMs are good at those aspects).
External tool call has an overhead. It requires a round trip into an external tool. It requires an LLM to run in agentic autoregression - it can't be used in prefill.
Which means that having native arithmetic capabilities is useful. Forward pass arithmetics are an LLM version of quick mental math.
An LLM can read "#define SILLY_TIME_CONST (3*20*60*60*1000)" and have "SILLY_TIME_CONST is 60 h expressed as 216000000 ms" already cached by the end of the line, before it even emits its first token.
When humans stall on these tasks, they reach for pen and paper, a slide rule, a calculator, etc.
Mathematica is overkill for arithmetic, in addition it's licenced and can cost a bit extra.
If an LLM were to reach for a light cheap arithmetic tool something like bc would be a good first stop - a CLI tool with a language that supports arbitrary precision numbers with interactive execution of statements.
Previously it used to run Python scripts, and may still do for more complex calculations.
In X years is it just going to be a thin OS-like layer where a majority of work is being handled by other "programs".
So, in essence, just like human beings?
Worse, this is really human beings trying to pretend that their AI is AGI.
Also big picture its hard to see it as some sort of self-improving intelligence if humans are hand crafting these paths and tools for it.
That is my hopeful ideal
But I don't as well, and I have some intuition about numbers that I would probably not have if I always relied on calculators. Would the same sort of thing apply to LLMs? I'm probably anthropomorphising here...
- color wheel
- oxidation reactions
- interpretive dance
- migratory patterns of curlew sandpipers
Whether one should is another question
now everything looks the same and i can no longer read on kindle.
What is it with LLM writing where it gives a smaller heading just before the main heading? Its nonsensical!