I've seen this going around my circle and found it very interesting. What I don't have though is a very useful basis for comparison. I'm curious if anyone here knows of any previous attempts at alternatives to back propagation? I think it'd be useful to me to understand better why backprop has withstood the test of time.
So the best part of backwards prop is that you need very little memory to complete the training which matters on GBs of data. FF seems to allow for more parallel computing but you will need to keep at least one more set of coefficients more in storage to deal with scheduling conflicts? Definitely merits more work.
How is this different from "skip-forwarding" of ResNets?