What does reasoning have to do with geometry? Is this like the idea that different concepts have inherent geometrical forms? A Platonic or noetic take on the geometries of reason? (I struggled to understand much of this paper…)
One of the references the paper provide is to this[1] paper, which shows how the non-linear layers in modern deep neural networks partitions the input into regions and applies region-dependent affine mappings[2] to generate the output. It also mentions how that connects to vector quantization and k-means clustering.
So, the geometric perspective isn't referring to your typical high-school geometry, but more abstract concepts like vector spaces[3] and combinatiorial computational geometry[4].
The submitted paper shows that this partitioning is directly linked to the approximation power of the neural network. They then show how increasing the approximation power results in better answers to math word problems, and hence that the approximation power correlated to the reasoning ability of LLMs.
[1]: https://arxiv.org/abs/1805.06576v2
[2]: https://en.wikipedia.org/wiki/Affine_transformation
[3]: https://en.wikipedia.org/wiki/Vector_space
[4]: https://en.wikipedia.org/wiki/Computational_geometry#Combina...
Since linear algebra is closely related to geometry[2], it seems quite reasonable that there are some geometric aspects that define their capabilities and performance.
Specifically, in this paper they're considering the intrinsic dimension[3] of the attention layers, and seeing how it correlates with the performance of LLMs.
[1]: https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc...
[2]: https://en.wikipedia.org/wiki/Linear_algebra#Relationship_wi...
Sure but this doesn't mean terribly much when you can relate either concept to virtually any other concept. "Reasonable" would imply one specific term implies another specific term and you haven't filled in those blanks yet.
Absolutely, in fact you can build the foundation of mathematics on this concept. You can build proofs and reasoning (for some value of "reasoning").
That's how dependent type systems work, search for HoTT and modal homotophy theory. That's how lean4, coq, and theorem proofs work.
If you remember at the foundation of lambda calculus or boolean algebra, they proceed through a series of transformation of mathematical objects that are organized lattices or semi-lattices, partially ordered sets (e.g. in boolean algebra, where the partial order is provided by the implication).
It would be interesting to understand if the density of attention mechanisms follow a similar progression as dependent type systems, and we can find a link between the dependent types involved in a proof and the corresponding spaces in a LLM via some continuous relaxation analogous to a proximal operator + some transformation (from high-level concepts into output tokens).
We have found in embeddings that geometry has a meaning. Specific simple concepts correspond to vector directions. I wouldn't be surprised at all that we find that reasoning on dependent concepts correspond to complex subspaces in the paths that a LLM takes, and that with enough training this connections becomes closer and closer to the logical structure of corresponding proofs (for self-consistent corpus of input and, like math proofs, and given enough training data).
Geometry problems have the nice property that they're easy to generate and solve mechanically, but there's no particular reason why a vanilla Transformer LLM would be any good at them, and you can have absolutely huge scale. (Unlike, say, the HumanEval benchmark, which only has 164 problems, which resulted in lots of accusations that LLMs can simply memorize the answers)
1: https://deepmind.google/discover/blog/alphageometry-an-olymp...
A weak illustration of this is this site[1], from an HN post a few months ago[2].
Since nobody can really say what a good AI department does, companies seem to be driven by credentiallism, load up on machine learning PhDs and masters so they can show their board and investors that they are ready for the AI revolution. This creates economic pressure to write such papers, the vast majority of which will amount to nothing.