So the initial “hey, this might be a good idea” implementation performs slightly worse than something that has had literally billions of dollars thrown at it?
No comparisons to AdamW were made.
In fact, this algorithm uses backprop at its core, but propagating through 0 layers.