The author is taking non-linear neurons.
This is, IMO, cheating.
I hope it's more than "presumably, non-linear neurons can approximate any non-linear function since they both have non-linear in the name".
1. A function can be approximately implemented as a lookup table.
2. It's trivial to make a neural network act like a lookup table.
Which seems to resemble the article but it's much simpler.
Point 2 assumes the neurons are normal non-linear ones. I'm not saying that's cheating, but I do agree with it being pretty obvious, at least from the right angle.
The only way you can make it substantially simpler is if you use a neuron whose nonlinearity makes it essentially a restatement of another result. For example, if the neurons are basically just Haar wavelets.
Make two neurons tied to Input that activate very sharply at the bottom and top edge of each bucket.
Use those to make a neuron that activates when Input is in the bucket.
Weight it so it adds f(x) to Output."
That's over 100 times shorter than the article. The method isn't as elegant since it needs two internal layers but I think it's pretty clear.
Is it wrong to say that the logical leap from "a neuron can go from 0 to 1 at a specific input value" to "neurons can make a lookup table" is trivial? Oh well.
(With "go from 0 to 1 at a specific input value" being the nonlinear part.)
It's also 6000 words long.
I'm saying it's not that hard.
I'm not saying the article is wrong or anything, I'm saying you can get to the same result MUCH faster.
"You can turn a neural net into a lookup table" should be easily understood by anyone that knows both of those concepts.
Edit: Like, isn't triggering specific outputs on specific input conditions the first thing that's usually shown about neural nets? If not a full lookup table, that's at least 90% of one and you just need to combine the outputs.
1. Use the function as the activation function.