You're badly missing the point. A neuron is not an ODE solver in the sense that you feed it some sort of mathematical specification of the ODE and it sits there and performs algebraic manipulations until there is some solution that pops out in the form of IEEE 754 floating point numbers. Neurons aren't Mathematica. A set of neurons
is the ODE solver. If there is a set of ODEs that describes some neural system in terms of the solution to the ODE, then the neural system profoundly
is an ODE solver.
You're now asking for a demonstration of something that can't exist because having already demonstrated a reasonable isomorphism between some ODEs and some neural nets, there isn't another layer of some sort of "REALLY REAL isomorphism", that is only a product of human cognition and the failures it has thereof. (I mean that in general, not specific to one person, humans in general are surprisingly good at creating distinctions where there shouldn't be.) A system is what it does. If the activities of some set of neurons can be accurately described as a solution to an ODE, then it is an ODE solver. If the activities of a set of neurons can be accurately modeled by a clock function, then the neurons are a clock corresponding to that function. If the activities of a set of neurons can be accurately described as an edge-detection algorithm, then the neurons are edge detectors. (Both of my other examples here are real too, by the way, I didn't just make them up for didactic purposes.)