The critique here is that the primary discussion is about fossilization rather than geological modification. A brutalist architecture building acts kind of like a granite batholith in sedimentary rock. And yes even then eventually things will likely be recycled but for definitions of the "last couple hundred million years" the argument can be made that there will be signs that something was making changes with intent.