Concrete, bricks and granite can also work pretty well... as the Pantheon in Rome proves.
The biggest issue with modern concrete construction is the usage of reenforced concrete using iron or steel rebar which tends to corrode quickly. Reenforced concrete seems to have a half life of ~50 years.
[1]: https://www.compositesworld.com/news/tprc-research-studies-v...
Basalt fiber rebar might be a better way to go, as it won't oxidize. Studies will be needed to find out if it actually is better in the long run.
The biggest problem is rebar spalling though, not the concrete.
Meanwhile, the kinds of beams and walls we make out of concrete wouldn't work if carved out of rock, they are only structurally sound because we put steel rebar in the concrete. But rebar expanding due to rust is how concrete usually fails. Plus using rebar prevents you from using salt water in your concrete, which is one of the ways in which Roman concrete might be better than ours.
In the northeast US, sidewalks frequently are in horrible condition after only a few years, because of the salt used in the winter. Sidewalks do not use steel rebar; they're just simple poured concrete. So why can't they make them with a better mixture that doesn't corrode so easily? (I'm guessing the answer is: replacing them frequently makes a lot of money for some concrete company that the town mayor is friends with.)