To me this is a bit exemplary of how science sometimes gets functional myopia. Conceptionally obvious: Playing with step sizes gives you higher chance of getting closer to the global (not local) optimum initially, because you can scout , but you want to get your algorithm working, so let's slap on a heuristic and let's first converge for local optima with learning rate decay. Then most people in the scientific community seem to forget (for example) about this heuristic, because it's Always Been There and direct all their focus on improving the status quo, forgetting that the assumption we put there is also a construction area. Lumpers and splitters, I guess. This sounds bitter, I'm actually happy Mr. Grimmer took this on!