I guess the whole idea of reinforcement learning is to "learn" how basic physics works from the observations

We can hardcode a solution in a simple example but how do you write a rule to drive a car from pure physics equations?