866 karma · joined February 18, 2008
And still miss my Lisp Machine. It's not that Unix is really that bad, it's that it has a certain model of computer use which has crowded out the more ambitious visions which were still alive in the 70s and 80s.
Much as the web (the Unix of hypertext) crowded out the more ambitious visions of what computational media for intellectual work could be (see the work of Doug Engelbart and Ted Nelson). That's a bigger tragedy IMO. Unix, eh, it's good enough, but the shittiness of the web makes humanity stupider than we should be, at a time when we can ill afford it.
There was a movement around 20 years ago to establish a field of practice called"software design" but I don't think it went anywhere. The problems remain the same though. http://hci.stanford.edu/publications/bds/
Also, evolution optimizes for survival and reproduction, not intelligence. While it may be possible to design an artificial evolutionary system that optimizes for superintelligence, I don't think doing so is trivial (or it would have been done).
Let's posit that your image of post-singularity non-friendly AIs is accurate. They are an immense and irresistible force, like a black hole. I can imagine something like that, but I have a great deal of difficulty imagining something like that that is also constrained by initial design decisions made by humans before the singularity, which is what the friendly AI people hope to achieve. To be fair I haven't read all their stuff yet and I'm sure they address this point.
First, I don't think that corporations are nothing but parasites, and said as much in the article. I did say that finance was parasitical. That was probably too strong. Allocation of capital is a useful function; the problem is that the allocators are helping themselves to a far larger cut of the proceedings than the value they produce (and I don't particularly want to argue that here, just wanted to clarify what I meant).
Second, the more central point about whether the Singularity Institute types are wasting their time and should refocus their thinking. It is true that not many people are thinking about the effects of superhuman AIs, and there's no harm and possibly some good in a few smart weirdos focusing on that, especially since corporations have been around and critiqued for so long.
However, we are so far away from producing anything remotely close to that level of AI, that I fear their work is ungrounded. It strikes me as fun quasi-SF rather than serious engineering.
But I take believe they are seriously interested in world improvement, so my suggestion to them in their work on "benevolent goal architectures" is to study how existing goal architectures work, that is to look at existing economic and social systems where disparate individual goals are coordinated into cooperative or conflicting action, and how the results are related to human goals and human flourishing.
For my part, I am in the process of trying to understand their friendly AI work better so I critique it from a more informed standpoint.
Actually there is a somewhat valid and interesting point in that essay, obscured by crap. There are differences between numerical computing culture and symbolic computing culture; people should know both. But from where I sit hardly anyone these days is familiar with symbolic computing while everyone either claims to be or wants to be a “data scientist”, that is, do numerical machine learning. Nothing wrong with that, but I have a hard time seeing numbers as the underdog in this fight.
The point was that Lisp has better tools for achieving good modularity.
...shows why individual choice fails to give us what we want, and why we need to rely on collective action rather than individual choice to take us to where we want to be.
Don't buy it on Amazon...
So, the implication that the NSA is much worse than the Stasi is wrong. But they don't seem to be any better, at least when it comes to data collection. That is not very comforting.