they could make steel cages, fill them with crushed and vibrated limestone (or another aggregate). Good steel for the cage would last a lifetime but not sure if it's cheaper in $$ or CO2 emissions. Basically they need to lift /drop a heavy thing that last generations and stacks well.

Concrete has the advantage of lasting "forever" in those conditions (rebar when wet ruins it, otherwise it last long, long time). Concrete also stacks perfectly as you can mold it however you want. Might bite the bullet and stick with concrete, just work on making it more efficient.

yep, hence the use of waste material wherever possible.