Ancient civilizations had fairly thin survive or starve margins. Civilizations that sit atop the best agricultural land and build vanity projects instead of armies and practical infrastructure don't tend to sit atop that land for long. While we don't have precise records nobody is perplexed about how they moved stones nor how they mobilized the population, we have many well understood examples of ancient civilizations doing these things. It's largely a question of what other special circumstances let them engage in these projects so prolifically when Baybylon and China built <checks notes> walls.