Hacker News Reader

This may be a silly question but - so rather than train a big network and hope a subnetwork wins the lottery - why not just train a smaller network with multiple runs with different starting weights?

View on HN

The larger network contains exponentially more subnetworks. 10x the size contains far more than 10x subnetworks (although it'd also take more than 10x as long to train).

Ah ok - so you are saying the explosion of subnetworks is higher than the explosion of training time - leading to a win.

In this case a positive benefit of combinatorial complexity.

Largely for the same reason people don't "just win" the lottery by buying every possible ticket (any more).

Reply on news.ycombinator.com