Fear of AGI is very real. But our proximity to AGI is marketing hype. In terms of existential crisis #2 and #3 are serious and already worth investing in mitigation. But #1 (AGI) is science fiction.
https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/#...
You can easily disagree with the warnings and fears shared in particular sci-fi, but to dismiss fiction as categorically irrelevant to our reality is just ignorant.
There is some important history here your comments suggest you are unaware of. One reason AI companies talk so much about AI risk despite it being bad for their bottom line isn't marketing hype, it's because many people in those companies are genuinely afraid. One could argue that this is because many of them have been exposed to the rationalist community- which could be (uncharitably?) seen as a doomsday cult obsessed with AGI risk. The founders of many AI startups including OpenAI were heavily influenced and active in this community.
It's easy to re-imagine science fiction as being closer to reality after the fact.
AI safety is an issue whether AGI is 10 or 100 years away.
By my reckoning, there are two driving factors in how the understanding and control of nuclear reactions affects our world: First, the huge step change in energetic density over chemical reaction, second the fact that you can cause explosive chain reactions. If there was a smooth gradient of progression towards the energetic density of Fat Man, the world would have been a very different place. If the science had panned out such that we couldn't cause a nuclear chain reaction, the bomb/energy source duality would never have emerged.
I'm curious if there is any fiction out there that contended with either of these factors prior to us realizing that the disparities between chemical and nuclear reactions are what they are. I've not read Alas, Babylon, and I'm not trying to deny that it had important influence, but I'm pretty sure we knew that it was possible to end the world via nuclear conflict prior to 1959. By your estimation, how did the book shape or prepare our society for the changes caused by nuclear technology? We're on the other side of the cold war and the conflict between capitalism and communism isn't a thing anymore, right? If we die in a nuclear fire, it will not be anything like, to quote wikipedia, "The explosion is mistaken for a large-scale US air assault on the military facility and, by the following day, the Soviet Union retaliates with its planned full-scale nuclear strike against the United States and its allies."
I'm not trying to be snarky, I'm trying to understand what I'm missing.
----
To me these claims of SF's prescience fall short, especially when it comes to the dangers of AGI. As an example, I have no mouth and I must scream, as wonderful as it is, doesn't have much to say about preparing our society for AGI. AM will not emerge from GPT-4o, and I'm pretty sure if AGI does happen it won't be through a super computer built to coordinate militaries. I enjoy reading SF, but I think its wider influence is as much miss as it is hit, and I think we can plot better courses when we treat SF as mostly entertainment. People who develop technology would be better served studying the history of technological development.
Now, to respond to your post-edit: I feel like my disconnect with your positions in this thread are around the importance of fiction. I think fiction is a minor muscle in the body that drives technology, if you'll allow the metaphor, whereas you claim "science fiction has been pivotal in both shaping and preparing our society for changes caused by technology." Perhaps you could explain what you mean more concretely? Another confusion I have: The claim of prescience and that the themes are very old seem to be at odds. If I develop an understanding of the dangers of runaway technology through a story about Genies, why do I need more modern work? What if I instead gain these understanding through actual history? The real world has so many more wrinkles than a story and so many more lessons to give about technological development than a story that I balk at words like pivotal. There is not pivot point, only hundreds and hundreds of articulations.
Anyways, I've written enough for now, and I don't expect you to follow up.
Why is sci-fi useful when we could just look at technology history? It's the range of imagination involved- exploring big possibilities and unintended consequences that are unlike anything we've experienced before. A lot of it is also noticing how historical patterns could play out in a new context, and sharing those ideas.
I am reluctant to list out prescience examples, because on one hand they seem so numerous I couldn't do them justice with just a few, and on the other hand people will disagree with the importance of them.
Overall- I feel like it is a simple fact that lots of tech people and inventors love sci-fi, grow up reading it, and then it influences what they think about and choose to work on. I suspect we wouldn't have had rockets or space travel when we did, if we didn't have so many victorian era books imagining space travel. The numerous billionaire pet project space companies we have now seem a lot like people trying to live out fantasies from reading Heinlein. Even little things like the devices we have- my Kindle looks and acts exactly like the e-book readers in The Next Generation, for example.
When my son sees old Star Trek episodes very little is remarkable to him except the big things like warp drive and transporters. The "advanced tech" they use day to day is all stuff that is a normal part of his life- 3D printers, tablets, ebook readers, video conferencing, mobile communicators/phones, etc. Same with the "Back to the Future" movies- they travel to a future filled with various electronic gadgets that are very similar to those we now have, but didn't exist at the time... and my son doesn't see how remarkable that seemed to me as a kid, when those didn't exist.
As a researcher in biotech, some of my current research is definitely inspired by sci-fi, especially biotech and nanotech from Neal Stephenson books.
Edit: found this interesting list on wikipedia https://en.wikipedia.org/wiki/List_of_existing_technologies_...
Where would I go if I wanted to learn more deeply than wikipedia about how hindu mythology talks about nuclear proliferation? Any directions I should search?