Continuous != differentiable.

The standard proof uses some functional analysis techniques but nothing too complicated to show you can get arbitrarily close to any continuous function with an NN. That includes things like step functions whose derivatives are not defined everywhere.