The weird thing is that big organizations just seem to give up when trying to fix these types of things.
Imagine what it will be like when ML is feeding the software values that are broken, wrong, or dangerous. How will they fix those problems? Especially when corporations often lose the people that wrote the applications, and the people behind them have little knowledge or skill.
I see the larger point is not that the overarching goal of the work being done may not be directly or indirectly influenced by a human. It’s not a reflection I think on who may ultimately be pulling the strings. In that sense, it does not require singularity to be meaningfully true.
But the idea that 10 billion people ultimately spending their lives dedicated to machines. Dedicated in the sense of it’s the first thing you touch in the morning and the last thing you touch at night. It tells you how to conduct your day, and mediates every human interaction you have meanwhile. The machine “masters” life like a mastering a record. It lays down the track of your days in concentric circles.
Who created the capitalist economy?
We're all in service to the machine already, and nobody is in charge.
Break the networks down far enough and you'll find the individual component's owners/creators. Even the protocols were created by specific people (and for specific reasons! The design philosophy of the internet protocol is pretty interesting reading) even if those people are no longer involved in the operation of the networks.
This was written in the throes of the Industrial Revolution. I suspect he was aware.
The minute an automated subset of the industry can perpetuate itself more efficiently than a partially human-powered counterpart, we're toast.