Toward an AI Physicist for Unsupervised Learning

The computer program BACON (1987) of Nobel Prize winner Herbert Simon was given the distances of planets from the sun together with their period of revolution and it independently rediscovered Kepler's third law, illustrating how far the positivism at work in "AI" can go. But Kepler's achievement was not determining a - straightforward - relation between two rows of numbers: it was to figure out which numbers should be related, and Kepler's real achievement was actually finding the right question. Incidentally, Kepler stated a fourth law relating planets to perfect polyhedra, and one wonders why this fourth law has not been rediscovered by computer yet...

-- Jean-Yves Girard: Locus Solum

So I'm still trying to grok this, but this could be a very important result if it could be generalized to the problem of meta-learning and model selection.

In that setting, the model selection oracle could access a shared knowledge pool of learned theorems that are biases about what kinds of models are better. e.g. convolutional nets are better than fully connected nets for vision tasks.

This could break us out of the diminishing returns we have seen with deep learning, by allowing us to better explore the space of compact model architectures, and develop shared biases about what is better. For example, learning the programmatic generation of Inception-like networks.

Bonus points if you want to add a blockchain connection, to decentralize the accumulation of the shared theorem base: Proof of work is figuring out what biases are true over some benchmark, which can be stored to a distributed ledger. Competing annotations lead malicious and noisy results to be penalized.

I got introduced to Max Tegmark's work last year. Got his book - Life 3.0 and Our Mathematical Universe