Search for "numerical integration" in the context of physics simulation/game engines. You can use https://en.wikipedia.org/wiki/Numerical_methods_for_ordinary... as a starting point.

From what I understand it's not just floating point errors, but due to the nature of approximating continues function as simple discrete steps. Linked wikipedia article has a graph demonstrating that with large steps the error accumulates way before floating point precision is a concern.

There are different numeric integration techniques with different tradeoffs. Stuff like Eulers method, Verlet, Runge-Kutte. In some of them overall energy tends to accumulate in others it gets lost both of which is wrong. Some of the more complex ones tend to behave a bit better, but then you get into the problem whether gains from each individual step being more complex outweighs what you would get from running more iterations of simpler/faster algorithm.