This is because, states at some size become immoral power machines deeply distrusting one another.
There is absolutely nothing rational about terraforming your own planet to make it less able to support you.
This is typically termed a "belief".
dictators make dumb-ass calls often, like Erdogan of Turkey making random (randumb) pronouncements about the currency and causing terrible inflation (or Saddam's "let just fight all our neighbors, Suharto's random evil, etc.).
Saddam I’ll give you but that also led to his removal and it can be argued Iraq didn’t have many good options as historically it’s been controlled by one or more of the surrounding nations so it may just not working realistically due to geography and ethnic strife
This line of reasoning only makes sense when compared with counterfactuals, which seems like a waste of everyone's time. It's easy enough to justify an arbitrary action as rational if you have no basis of comparison.
Anyway, "rational self-interest of the state" is not the same thing as "rational". There are other ends other than self-interest of the state—for instance, self-interest of the constituents of the state, or self-interest of humanity. All states put their own existence before the welfare of their people. A state is not a natural thing outside the vying of capital to institute economic stability for the ends of its own empowerment.