https://en.wikipedia.org/wiki/Arrow_of_time
If you go to the 'overview' section, there is a nice snippet explaining all this. It points out that gravity works fine in a universe that is running backwards, however entropy is decreasing instead of increasing.