Yet religions have a vision of the workings of planets (and the solar system in general) that is much further removed from reality than astrology.

Yes, you could say it's a metaphor, a symbolic vision or something else rather than something to be taken literally in religion, but the same could be said of astrology.