The Unreasonable Effectiveness of LLMs in Mathematics

A summary (not by AI):

Hadamard asserts that no mathematical discovery is purely logical. The unconscious mind ... played a crucial role in the development of rigorous mathematical arguments. This role, and the handoffs between the subconscious and conscious minds, were distilled by Hadamard into the following framework for mathematical discovery:

- Preparation (primarily conscious)

- Incubation (primarily unconscious)

- Illumination (primarily unconscious)

- Verification (primarily conscious)

DeepMind's AlphaProof is too "conscious", missing Incubation and Illumination, and hence does not work well. In contrast, LLMs are more "unconscious", emulating Incubation and Illumination better, and thus have better chances to make math discoveries, at the risk of producing false results.

However, LLMs that reason in languages are still not "unconscious" enough; the Looped Language Models (by ByteDance) can reason in an even more unconscious way, aligning better with Hadamard's observation that "in addition to being non-rigorous, unconscious thought is often not even interpretable ... all mathematicians think without language or precise symbols, and many do not even use clear images", leading to higher reasoning capabilities.

A combination of the two approaches (AlphaProof and LLM) seems to be able to close the loop of Preparation - Incubation - Illumination - Verification in math. In addition, this framework is promising in "any domain that can culminate in a Verification step", and LLM may do the unconscious "Incubation - Illumination" part in many domains in addition to math (e.g., physics), but the "Verification" part differs across domains.