Happy to hear thoughts from any perspective; whether that be drawing from evolution, systems theory, economics or beyond.
There will be some neat technology leading up to it, though. At least the next 10-20 years. So that'll be fun. Maybe a couple new web frameworks people will get excited about.
[1]: https://www.nationalgeographic.co.uk/environment-and-conserv...
[2]: https://www.scientificamerican.com/article/humans-are-doomed...
1,000 years ago most people lived their lives as much as they had to and lived their fantasies as much as thy could. 1,000 years from now, the same will apply.
The especially special standout of our period in time is that we think our fantasies are shared more widely than they are. The delusion isn't different in degree than people expecting the Biblical Apocalypse 1,000 years ago, and our descendants will have their own delusions they'll hold with as much fervor and as little reason.
I don't think so. Humanity isn't some uniformly distributed / synergistic / perfectly aligned entity. It's a mess of individuals, small groups and large groups with different incentives each one of them pulling and pushing in a different direction.
At some point, collective intelligence will be used to explicitly create better collective Intelligence systems. I believe that if humanity reaches that point, our species will survive and thrive. If we don't, we'll go extinct.
But I'm optimistic and think reaching that point in the near future is realistic. If only a company with enough resources realizes that they can combine the brains of their employees to create something more intelligent, they would have a huge competitive advantage. And that's worth working on.
If you're working on collective intelligence systems, too, please reach out.
Phrased less cynically, we are all building our own little future, and humanity is the some of our constituent parts.
To also give a more pragmatic answer to your question, I think we are progressing fastly towards a global society, with centralisation of both private sector and state power in the hands of a few elites. States are generally becoming more authoritarian, while companies are becoming ever larger and more complicated. We are already in a situation in which the largest companies can take on smaller countries in "asymmetric warfare", and I think this will just escalate with less borders and a potentially shrinking cake due to climate change and changing demographics.
living paycheck to paycheck for most. Hopping their children will be in first category.
Just trying to survive for the most unfortunate ones.
In short, we're maximizing the math/fun utilization per brain. This has all kinds of side effects, like galactic colonization.
For some amazing statistics on "what we're building towards" I recommend looking at Hans Rosling's TED talk, where he describes how civilization is becoming better (over time) by every single measurable metric.
CO2 concentration in the atmosphere is increasing. Fossil fuel, metals, and other one-off resources being depleted at exponential rate. Biodiversity is collapsing -5 exponential rate.
We are on our way to a grim future by all measurable metric i can think of. The metrics you have in mind are great for the here-and-now, but a blink in the longer timeframe of our species.
This goal has driven all of life. We were born with the impulse, it drives evolution itself. It's why we reproduce. It's why we invade new lands. It's why we dream of Dyson Spheres. It's why we dig up fossil sunshine and will continue that project until it kills us.
But some other life will get the same idea and carry on after we fail.
https://www.quantamagazine.org/first-support-for-a-physics-t...
Things that it could probably do: Write song lyrics, screenplays, identifying drug dealers, things political parties can agree on, gauging your mental health similar to how Garmin watches can gauge physical health, finding MH 370, pass a typical tech interview.
Also if you play with AI as it stands today, you'll notice some characteristics. It reminds me a little of Vision in WandaVision. It's adorably naive in some things. It has a certain "dialect", probably depending on how it's trained. It reminds me of a 14 year-old American.
The simplest(!) problem to solve before AGI becomes anything other than a civilizational footgun is "rigorously defining perfect morality." After that comes the task of programming something that we have very fuzzy, often misleading intuitions about (i.e. general intelligence). And programming it perfectly, of course, since any deviation from the defined spec will trigger the footgun. And since the thing will have to be able to self-modify, we have to assume an adversary who is by definition (likely orders of magnitude) more intelligent than you possibly trying to subvert said programming.
I find Yudkowsky's writing rather illuminating in this regard (for example, https://www.greaterwrong.com/posts/4ARaTpNX62uaL86j6/the-hid...) although he remains far more optimistic than I am.
We're programmers, considering the creation of what is essentially a god. As if an algorithm merely destroying a life through a bug were unheard of, or too small stakes.
Some interesting examples; https://docs.google.com/spreadsheets/d/e/2PACX-1vRPiprOaC3Hs...
One notable case from that file: 'A researcher wanted to limit the replication rate of a digital organism. He programmed the system to pause after each mutation, measure the mutant's replication rate in an isolated test environment, and delete the mutant if it replicated faster than its parent. However, the organisms evolved to recognize when they were in the test environment and "play dead" so they would not be eliminated and instead be kept in the population where they could continue to replicate outside the test environment. Once he discovered this, the researcher then randomized the inputs of the test environment so that it couldn't be easily detected, but the organisms evolved a new strategy, to probabilistically perform tasks that would accelerate their replication, thus slipping through the test environment some percentage of the time and continuing to accelerate their replication thereafter.'
An AI does not need to be malevolent - it merely has to be powerful and poorly aimed. The more powerful it is, the smaller the margin for error in the aiming. And with an "infinitely powerful" AI, we basically have to perfectly solve morality or else it will optimize for a future we do not want.
I would say what all animals naturally work towards: survival and to perpetuate the species.
Some larger-scale themes: Getting off of the planet. Reducing poverty. Renewable energy. Knowledge available to everyone.
This is fiction. We aren't striving for anything.
With the rate of change in technology, the limits we have now that make surviving off world will surely dissolve over the centuries
I think, like much of last century’s science fiction, it doesn’t stay fiction for so long
Almost every human who has ever lived has died unable to solve fundamental problems of how to prevent death due to natural causes or disease. There are problems which- so far- are beyond human intelligence.
While I can't rule out the possibility of people living offworld one day I also can't rule out the possibility we'll hit a limit of human intelligence or organizational capability.
The far future an individual human is perhaps unlikely to be able to contribute much - perhaps as you say, we’ve reached a plateau of development
And in the past individuals had less mobility to contribute to what they wanted
But right now we’re in the middle of an exponential explosion of development and we can’t get enough smart motivated people to power it
So right now it’s possible to get involved in so many exciting projects and make a decent contribution - including making humans multi-planetary if that is your dream
We've abandoned all big science, technology and engineering projects in favour of comfort and entertainment.
I'll say this for the USSR, they had big goals at least. That's all gone now because people don't want it. They want a slightly larger house or a quicker rate of change of fashion or more extravagant sports contests (eg the Iraq war 3)
I totally disagree with your second one.
Unless there’s some sort of major reboot, the button for which I suppose we are busily groping around for.
Any of several buttons.
Interconnected buttons.
The buttons of DOOOOM!!!
Many people are nostalgic for USSR’s big goals, except for those who did not share the ideas or means to achieve them (or were simply killed in the process).
People less willing to work, vote, socialise, or just plain live.
https://www.huffpost.com/entry/elon-musk-mars_n_3359773
or mayhaps a shift in consciousness
And that drives the process of evolution(biological, economical, technological, others), towards more complexity.
So basically there were some rules made up for the game we'll be playing during our lives. Those rules get indoctrinated within us, then we try to fulfill our inner desires and imagined goals using those rules, following them, breaking them, sometimes reinterpreting them, etc.
On the plus side, if we look back several hundred years we like what we have today better, so hopefully in another few hundred years things will be even better.
I really hope it won't be the Zuckerberg version, and I'm not talking about weird crypto schemes, VR/AR goggles or Pokémon Go. I mean the original idea as described in 80s sci-fi novels, Ghost in the Shell, Matrix, Avatar, etc: Brain implants that let you plug into some kind of fully immersive VR world (or alternatively, a remote-controlled robot body in the real world) and gives you an experience indistinguishable from reality.
I don't think there are many people that consciously want this future (except maybe furries, trans folks and tech billionaires), but there seems to be a remarkable number of both push and pull factors that move us into that direction:
- The technology seems to become increasingly feasible (in the next decades) : Biotechnology is already one of the most active and well-funded areas of research and the pandemic will only increase this development.
- A significant number of common "first-world problems" seem to be about body image: Beauty standards, gender identity, but also disabilities or age. I think the ability to design your own body as you see fit or to swap it out as we are changing clothing today will have an allure that should not be underestimated - especially in an individualistic and narcissistic society.
- Humans are overtaken by machines in an increasing number of areas, even without some kind of killer AI: Today, our machines can travel to other planets, explore physically plausible 4D spaces, trade in milliseconds or simply skip arbitrary amounts of time in standby. Only humans are stuck in the old world. Turning people into cyborgs might be an attempt to bring humans back into the loop here.
- Most of us are already spending a significant part of our lives in virtual worlds - social networks, chats, etc - but we do so in an awkward way that requires us to constantly divide our attention. Calls for "digital detox", going analog, etc are numerous but don't really seem to lead to much. Going the opposite direction - going fully virtual - could be an attempt to solve the attention problem.
- The current streak of "bad years" is unlikely to stop anytime soon and the real world will likely become a more unpleasant place than it was during the end of the 20th century: Either we'll be hit by the full force of climate change - or we'll voluntary restrict ourselves to avert the worst. In either case, the world will be less inviting and less free to explore than it used to be. So the desire to escape to a different world will likely grow stronger.
- Governments and corporations will probably be in favour of a Metaverse as at least a centralised version of it would greatly simplify social control.
I think there are some fascinating aspects to all of this, but there are also lots of obvious ways how this could turn into a dystopia. At the very least, if it should come to this, "decentralisation" will be a fundamental necessity and not just a crypto catch phrase - otherwise, we can quickly end up in a world where FAANG pushes "bug fixes and performance improvements" to your brain and where you'll be physically unable to sing "Happy Birthday" without a Spotify subscription.
Zoom forward to life. Life is more interesting. It tries to do things. You had early creatures that evolve arms and these arms were also their throats and mouths. You had creatures invent all kinds of weird senses, like being able to see electricity. There was the dinosaur era where creatures wanted big powerful bodies. Sharks were megasharks.
At this point the most complex thing in the Solar System was life. Not the sun or planets or some whirly space dust. Life was highly unpredictable and interesting.
Now fast forward to humans. Humans learned tools. They stat dumped claws, poison, and monkey muscle. They became really focused, eyes in front of their heads instead of the sides. Just became really good at making tools and using it as an extension of the body. Animals can make tools, but it doesn't meld into the brain like it does with humans.
Tools opened up a whole different layer of complexity. You get spears, then swords, hammers, houses, concrete, so on. You use tools to build more tools.
Somehow we became extremely complex creatures. And we've learned to deal with that complexity as well - statistics, systems, game theory, theology.
18th century, we invented economics. Simply put, wealth wasn't gold and silver, it was production. The idea that if humans were productive, they'd be wealthier. Wars slowed down a lot because you didn't have to kill each other for gold. Equality improved, because you'd have more productivity with willing employees than you would from slaves. And we got into this grand idea that we should just make everyone productive! Education, health care, free people in jobs, emancipation. This is the birth of modern civilization.
20th century, we got really good at tools making tools. Now you have programming. Punch cards, then Assembly, C, operating systems. You use an operating system and a programming language to make better operating systems and languages!
Then CAD. Build better processors. Build better software with these processors. Build better hardware with the better software! What is this wizardry?
And we've had these loops in every other field. We made humans last longer. Life expectancy was less than 50 in 1900. So humans grew older, became more experienced, became better doctors, better teachers and professors. We've had advancements in farming, so things were not as doomed as Malthus predicted.
While software and hardware is slowing down, this tools making more tools opens new doors. Genetic engineering is close. We're hitting advances in energy, efficient cars, rockets. That thing with creatures spending millions of years engineering themselves, we should be able to do that in a few years.
If we ever do figure out space colonization, we'll inflict all kinds of chaos. Stars are predictable over the span of billions of years. But give humans ten thousand more years and who knows what we could do to the stars?
tl;dr: We're always racing to become more complex and interesting, and humanity as a whole will continue to be unpredictable.