Hacker News Reader

This makes me wonder, if a lottery ticket network is roughly 10% of the size of the original network, then I would expect that statistically you'd have to train a randomly initialized verion of it 10 times to get the same result once. Is that true? Or is the actual topology of importance here?

~No. Suppose that 5 of 50 channels in a particular network layer made up the 'lottery ticket' for that layer. The number of 'combos' of 5 channels that were trained at the same time is 50C5 (i.e. ~2million [0]).

Whereas training 5C5 (=1) 10 times only gives you 10 chances to get the right 5 together.

At least, that's one way to think about it.

[0]: : https://www.mathway.com/popular-problems/Finite%20Math/60182...

Follow up from ICLR really elucidates some of the questions raised by this: https://eng.uber.com/deconstructing-lottery-tickets/ https://arxiv.org/abs/1905.01067

thanks for this. I wonder if the biological pruning process the brain uses works similarly.