Ooo, great question. In both cases, we define a function using a sum over an infinite number of infinitesimal things.

In the GP case, those things are random and independent of each other, so the central limit theorem applies and we get a simple Gaussian.

In the ODE case, the infinitesimals are deterministic and depend on the previous ones in the sequence, so the final answer is deterministic and impossible to compute exactly in general.

You could also use a GP to model the dynamics of an ODE, and this was done recently: [https://arxiv.org/abs/1803.04303] although the drawback was that they couldn't train the model by backpropagation.