In light of this, would you recant this point?
In light of this, would you recant this point?
But the naive interpretation of this leads to a singularity, and the fact it leads to a singularity shows that that interpretation is wrong and that there should be a limiting factor somewhere, the article suggests some of them.
As for the black hole, it is hard to argue using that analogy since you don't want to get close to a black hole for very well known reasons unrelated to the singularity, but let's get into sci-fi mode and say you could, it may be risky but it may also be worth it. The thing is: from the point of view of the travailler, you never cross the event horizon, and you get to know what's inside the black hole as the event horizon recedes from you, maybe you will never get back, but maybe there are greater things in there. And we are already trapped anyways as we are going one way through time, just as in black holes, things go one way through space, so maybe it is a risk worth taking.
> "but maybe there are greater things in there."
Probably not though; of all the arrangements of matter and energy possible, most of them are hostile or indifferent to Humans. Very few are beneficial to humans. Even Venus and Mars which are roughly Earth-sized rocky planets with atmospheres which orbit near the Goldilocks zone of a yellow dwarf main-sequence star, are still incredibly hostile to humans and they're far closer to Earthlike than most things in Space.
I think the most likely scenario where we make something much smarter than us is bad. I don’t agree with all of what Yudkowsky says, but “Nature is allowed to kill you” is something that I think not a lot of people grasp on a real intuitive level. We are totally allowed to invent technology that gets out of control and kills us all. It could have happened with climate change if the earth had stronger feedback. We’ve been lucky so far, but as humanity gains power, we need a corresponding increase in our ability to coordinate and be careful to match.