Almost any ML algorithm can be thrown at CIFAR10 and achieve ~60% accuracy; this ballpark of accuracy is really not sufficient to demonstrate viability, no matter how aesthetically interesting the approach might feel.
Almost any ML algorithm can be thrown at CIFAR10 and achieve ~60% accuracy; this ballpark of accuracy is really not sufficient to demonstrate viability, no matter how aesthetically interesting the approach might feel.
If he invents the new back propagation, an army of grad students can turn his ideas into the future. Like they've done for the last 15 years.
He's posting incremental work towards rethinking the field. It's pretty interesting stuff.
Edit: grammar
But "any ml algorithm" isn't the point. It's a new optimization technique and should be applied to models/architectures that make sense with the problems they are being used on.
For example, they could have used a pretrained featurizer and trained the two layer model on top of it, with both back prop and FF and compared.
Making the assumption that weights/embeddings produced by a backprop-trained network are equally intelligible to a network also trained by backprop vs. one trained by this alternative method.
That said, I am also short-term bearish on backprop-free methods (although potentially long-term bullish).