Hacker News Reader

Back prop and attention are just extensions of interpolation.

By that logic it's all "just linear maths".

Back prop requires and limits to analytically differentiable in a normal way.

Attention is... Oh dear comparing linear regression to attention is comparing a diesel jet engine to a horse.

It's all just a series of S(S(S(....S(0)))) anyways.