A Frightened Optimist On The Future of Humanity

And yet you are being downvoted for fighting common bullshit, what a world we live in!

You have to look at the longer term with technology, so far it's been an overwhelmingly positive force for humanity. I personally think it's the greatest thing ever, but we may one day invent the thing that ends us. Maybe that's AI, maybe we unlock a power in the physical world that we can't be trusted with. For example if anyone could build antimatter bombs in the garage, terrorists or other discontents would have used it to end civilization. There's no law of nature that says great power must always be hard to come by, and I wonder if that's not the answer to the Fermi paradox. Civilization might be a short-term blip, an unstable state on cosmic timescales.

H.G. Wells ends the Time Machine with: "for the question had been discussed among us long before the Time Machine was made - thought but cheerlessly of the Advancement of Mankind, and saw in the growing pile of civilization only a foolish heaping that must inevitably fall back upon and destroy its makers in the end. If that is so, it remains for us to live as though it were not so."

I definitely agree with you. But i cannot stop thinking about the dwarfs in the lord of the rings that went too far, dug too deep, potentially due to the rhetoric youre having. That was their doom.

Its a book, just a book. But i keep on thinking about it because i find it might also apply to humans and technology.

I think it is fair to see both sides of technological advancement. For every positive use of technology there is often some negative use. I think of nuclear power vs. nuclear weapons. No one thinks we should stop investigating chemicals and diseases just because someone could use the information to create chemical weapons or biological weapons. However, we do have social and political tools to attempt to deal with these negatives, whether they be conventions or treaties. As a more recent data-point look at all of the hubbub caused by social networks and their use by opposing countries to destabilize political process. We are going to make real changes to technology to attempt to mitigate those issues.

We need to think about the negative consequences of gene editing, robotics, super powerful AI, etc. not to overshadow the positive benefits but to help remind us to create the tools necessary to mitigate those negatives. If the only possible ramifications were good then we wouldn't need to be having this conversation.

It reminds me of interviews with Steven Pinker and his opinion on potential negative outfall from AI research. He claims that engineers are generally good-willed and they try to do the best they can, e.g. they don't design bridges that will fall. Yet bridges fall down sometimes, occasionally quite soon after they are built. I'm glad there is some external oversight on issues that are potentially world-changing, just to have a sober second opinion.

You could say that Eisenstein was doomsayer about nuclear fission.

And we did came quite close to doomsday scenarios, cold war, but also false-positives in automated detection defense systems.

That flies in the face of the history of progress. Maltheus was wrong for the last 200 years of doom and gloom predictions. Why would you expect differently now?

In actual fact, the amount of resources (atoms, power) in the Earth are orders of magnitude beyond what humans need for thousands of years, even assuming exponential growth. And the carrying capacity of the Earth has grown as fast or faster than population, explicitly because more people means more innovation and more capital which means higher carrying capacity. What's more, the per capita quality of life grows linearly with population, meaning a quadratic increase in available resources overall. More people = more prosperity. It's not a zero-sum game.

Inoculate yourself against the Malthusian ideology of the Club of Rome. The world is not ending, as much as their anti-human agenda would want you to believe.

Well its not an impossible argument to understand from a game theory perspective of civilization.

I don't agree with it. Much as i don't agree with the Dark Forest premise set forward in the Three Body Problem series, I thought it was very interesting nontheless and worth spending quite a bit of time musing over. As with Nick Bostroms thoughts.

Just because we disagree with a concept on ethical or practical or whatever reason doesn't mean we should be discussing it or that we can gain something from discussing it.

At least he can argue his side, unlike the politicians constantly pushing for a more and more draconian state.

As opposed to hoping for voluntary self-restraint and then washing our hands of the consequences? I agree with his response:

> Yes of course surveillance capabilities increase the states’ ability to suppress the restless populations…

> With some of the developments in nuclear technology, we now have the ability to wage wars that would have immediate planetary implications. So our technological prowess is galloping, whereas our wisdom and ability to peacefully cooperate with one another seems to develop at a slower pace. So, I think we need to spur on this cooperation horse to make sure that we can keep up with the galloping technological horse.

> If we think about this as a race between our growing powers and our ability to use those powers wisely, it seems unclear who will win that race.

What is stopping people from refraining from unethical use of technology, right now? Nothing. Yet many people still don't refrain, and think if something profitable can be done, if people can be manipulated or used in a certain way, then it's "smart" to do that. What'd be the the problem with not having growing disparities of power, and first becoming decent societies, and then getting to play with all sorts of fancy tech?

> If machines produce everything we need, the outcome will depend on how things are distributed. Everyone can enjoy a life of luxurious leisure if the machine-produced wealth is shared, or most people can end up miserably poor if the machine-owners successfully lobby against wealth redistribution. So far, the trend seems to be toward the second option, with technology driving ever-increasing inequality.

-- Stephen Hawking

And that inequality and making people superflous will lead to the possibility of atrocities that will be committed because they can be over in an instant, with nobody but perpetrators left to accuse them. Even right here on HN I had a short conversation with someone who earnestly thought we will and should be replaced with GAI, the next step in evolution so to speak, and that people should work towards that in secret, because people wouldn't accept being superseded by something much "better". I don't know what would lead to such an outlook, but it exists.

Just corporations doing mass surveillance isn't an absence of mass surveillance, the difference would be governments (a singular state is not mentioned, nor a police state) doing that too, preferably in working democracies, the achievement of which is part of the above "using our powers wisely" (and choosing wisely, whom we empower to do what on our behalf).