There was a nice paper about five years ago that also found that the best hyperparams were in the bounty of divergence, even for 'real' imagenet models.