Don't quote me on this but I think the limiting factor is that gravitational pull needs to be stronger than the expansion of the universe.