> Whether scaling laws hold or not, is up for debate.
The scaling laws have broken: https://arxiv.org/abs/2210.14891
308 karma · joined March 1, 2014
The scaling laws have broken: https://arxiv.org/abs/2210.14891
Evolution is a slow algorithm, but it had access to an absurd amount of compute (all neuronal organic matter on Earth) and environment simulation (all of physical reality on Earth) when discovering us; so the discovery of the algorithms/architectures/principles in our heads shouldn't be viewed as trivial.