You are nitpicking, and completely missing my point. Let me rephrase it.

The original article is about neural networks being able to represent any function (for some definition of any).

I was just pointing out that there exists much cheaper ways of representing any function. Therefore the article seems very unexciting to me.

Btw, you have chose L^2(R) yourself and then used that to show that there are interesting function not in the space you have chosen, quite a circular argument.

Since the article on neural networks never mentions functions defined on a infinite domain, one can easily take L^2([0,1]) or L^2([0,Lambda]) up to some cutoff Lambda. I would say that all non-pathological functions you can think of are there!