Would they really?

Sure, having an old Colosseum or Aqueduct to look at is nice, but do you really want to live in a place where every single square inch of useful ground in the whole country is occupied by an ultra-resilient building that somebody thought would be useful 2000 years ago?

We have all of these examples of nice long-lasting roman architecture because those are the examples that survived. Romans didn't live in the Colosseum and the aqueducts. The Romans built hundreds of thousands of other buildings that fell down on their own or were demolished for any number of reasons over the millennia.