They had stuff like theatres, temples, and aqueducts that still stood after being utterly neglected for a millenium, whilst their fancy new churches took centuries to get built and needed constant maintenance. It’s easy to see how this would seem wonderful, or how they would take ancient Rome’s engineers and architects as models.

Also, we tend to take things we’re familiar with for granted, and overlook them when comparing different societies. So of course, ancient civilisations were terrible for other reasons, but still...

Roman artists used vanishing point perspective, which to medieval artists -- who had no grasp of perspective at all -- was pretty amazing. What the Romans didn't do was use vanishing points systematically -- they would have multiple contradictory vanishing points in a single image. Brunelleschi's invention of 3-point perspective was much more systematic than what the ancient world had done, but it was very much inspired by the desire to reverse-engineer Roman techniques.

We have been trying to reverse engineer Roman cement for quite some time. There are quite a few things that past civilizations did that are impressive by today’s standards and then lost to time.