Now AI is beginning to devalue knowledge work. Although the limits of current technology is obvious in many cases, AI is already doing a pretty good job at replacing illustrators and copy writers. It will only get better.
Who owns the value created by AI productivity? Ultimately it will be shareholders and VCs. It’s no surprise that the loudest voices in techno-optimism are VCs. In this new world they win.
Having said all this, I think Ilya’s concerns are more of the existential type.
If history is any indication not really. There's an obvious dialectical nature to this where technological advance initially delivers returns to its benefactors, but then they usually end up being swallowed by their own creation. The industrial revolution didn't devalue labor, it empowered labor to act collectively for the first time, laying the groundwork for what ultimately replaced the pre-industrial powers that were.
Only some types. It caused a great number of people to be employed in manual labour, in the new factories. The shift to knowledge work came much later as factory work (and farming) became increasingly automated.
If the Wright brothers had focused on safety, I am not sure they would have flown very far.
Not unless you connect a machine gun to your neural net.
Otherwise - we are talking about a chat bot - yes if there is no safety - it will say something racist - or implement a virus for you that you would have had to search 4chan for or something.
None of this is any more dangerous than what you can find on the far corners of the internet.
And it will not bring the end of the world.
Imagine a chatbot that can reward people with becoming a billionaire via the stock market if you just do what it says. Imagine a chatbot that can pay your colleagues in solved bitcoin block hashes to kill your children if you don’t do what it says to connect it to more systems. Imagine a superintelligence that can inductively reason about gaps in a missile defense system and dangle that informational carrot to generals in exchange for an unfiltered ethernet port.
There is a great A24 movie called “Ex Machina” that explores this concept.
Superintelligence is inherently dangerous to the human status quo. It may be impossible to develop it “safely” in the sense that most people mean by that. It may also be impossible to avoid developing it. This might be the singularity everyone’s been talking about, just without the outcome that everyone hoped for.
Anyone claiming 100% certainty about how safe or unsafe it will be is either delusional or trying to sell something.