Banning self-recursive improvement in AI models?

He is of course 100% correct. There is no way to stop RSI without truly global cooperation and enforcement. And the incentives are such that global cooperation will never happen. So what Klein and others are really discussing is depriving the U.S. of the benefits of RSI while other nations race ahead. Even if China and the U.S. were to officially ban RSI, the compute would simply migrate to friendlier jurisdictions. In any case, enforcement, as Cowen correctly points out, would be nearly impossible.

None of this is to say that labs and researchers shouldn't take alignment extremely seriously and dedicate major resources to it. But, again, incentives. The lab(s) that allocates more resources to "intelligence" than alignment will have a compounding marginal advantage. And they all know that and are acting accordingly.