Hacker News Reader

Isn't that just like a Taylor series?

Yes - or Fourier series or other decomposition into a sum of orthogonal basis functions.

I don't think orthogonality is important here. Taylor series basis functions are not orthogonal, and nor are ReLUs.