They could not fathom how far the objects in the sky were from them.
They had to estimate distance to the Sun. From that they could have a rough scale... admittedly, ~ 3e7 (seconds in year) * 3e8 (meters per second, speed of light) * 4 ~ 4e16 m (approx. 4 light years) is significantly bigger than 1.5e11 (Earth-Sun distance), so it's still hard to accept - interstellar distance (Sun to a closest star) is about 300 000 times bigger than Earth-Sun distance...