> If you step way back, the simple fact that the Earth still exists in its current form tells us that no recursive process in the multi-billion-year history of the planet has ever spiraled completely out of control; some limit has always been reached.

I dunno, I mean, humanity has made some serious changes in a very short time. We're pretty lucky that the climate's sensitivity to CO2 isn't worse than it is, or we could have taken ourselves out already. Same as if we had had a nuclear war.

And AGI doesn't need to completely destroy the earth to be really bad for humans. Just taking over a lot of the resources we need would do the trick.

> It’s conceivable that some sort of complexity principle makes it increasingly difficult to increase raw intelligence much beyond the human level, as the number of facts to keep in mind and the subtlety of the connections to be made increases.

There's a whole lot of "it's conceivable" in here, which seems to me to be a bit of a coping strategy. For humans, the problem with our biology is that our heads kinda have a physical limit on their size.

The idea that not only is it possible to make machines that are stronger than us, tougher than us, more precise than us, and faster than us, but also just straight up smarter than us in the general sense (instead of just at math or chess or whatever), is not outlandish at all. They might be less efficient and take a whole lot more power, but that sort of thing hasn't stopped us before. We just find ways to get them more power. I accused the author of trying to cope with fear of the development of super-intelligence by figuring out happy scenarios where it will be harder than we think to create. I have done the same thing, but I think we do ourselves a disservice by not facing the potential for this to be a real problem head-on.