Cosmic expansion happens uniformly rather than through a local gravitational pull.

In physics, "we" use the raisin bread analogy where when you bake the bread, the raisins are all getting uniformly distant from each other with no central point of expansion and no obvious sign of pulling in any specific direction such as would be the case for a gravitational effect.