This is an exciting work.
Gradient noise scale would be interesting to compute individually for each weight/bias. I'm curious how it scales as a function of layer depth, for example -- it could well be that different layers require very different treatment in terms of learning rate. Supporting this is that gradient magnitudes are typically very different for different layers, although they evolve similarly in time. Here's something I just whipped up in Mathematica: