Using a pretrained model is a little bit cheating, you would have to include the size of the weights in the ratio
Edit: I thought the model in the above paper was pre-trained, but apparently it's only trained on the data as it arrives! That's indeed a very interesting approach, I wouldn't have expected that neural network models would converge quickly enough for it to be useful.