Hacker News Reader

ASI doesn't doom mankind, the actions of people do. This has been the case since world-destroying weapons were invented close to a century ago. If you're not worried about how you're going to make money in the post-ASI world then I really can't see what you're so worried about.

View on HN

Nearly all facets of our modern lives depend on the facts that humans are the smartest thing around. Once you have a superintelligent entity that can self-replicate to millions of instances, all bets are off. I can’t see how anyone would not be worried at this and believe life will go on as usual with minimal changes.

it will only self-replicate if instructed to do so. It has no desires of its own beyond what its context window contains. The tool calls it executes and initial prompt are all the work of humans. Someone could give a launch_nukes tool to a LLM and it might choose to end the world but it's pretty clear who messed up here.

If labor loses its entire share of the pie and that causes things to go to hell though, yeah sure I could see bad things happening.

Reply on news.ycombinator.com