I feel the human race has made a rather amusing step forward to rule-by-algorithm. 30 years ago this would be dystopian, today its a byproduct of a mish-mash of open source libraries.
IF ((I think that he/she is hot) && (he/she thinks I am hot))
THEN
date();
I do understand why people don't like it. Everyone feels that an algorithm could screw them over in a situation where a person wouldn't. It's the driverless car argument. And we all know the response, that humans aren't perfect either and we have better metrics and tools for understanding and fixing machine mistakes than we do human mistakes. The same is true for algorithms that control Tinder, credit applications, health insurance rates, et al. They aren't always right and it does suck when you can't make an appeal to human compassion. But humans can be just as wrong and just as obstinate, if not moreso. At least with an algorithm you know where you stand.
I, for one, welcome our algorithmic overlords. As a hacker I've always been able to figure out how technical systems work. Other humans? Not so much. If anything, this new world will play to my strengths and to the strengths of those like me.
Just imagine a world where you'd never have to hear "because I said so" and "becasue fuck you, that's why" ever again. A world where for everything that goes wrong, there'll always be a clear way to fix it and some clearly identifiable someone liable for the mistake. A world where being handsome is no more an advantage than having blue as your favorite color. Etc.
Maybe it's not perfect, but it would most certainly take away power from people that don't deserve it. I can see why the "powers that be" would try to convince us that this is "dystopian".
And when the AI does reach human level (a long time from now), we may be no more capable of understanding its motivations than we are of understanding those of other humans today.
Not if there are incentives for the algorithms to be subject to public scrutiny.
> And when the AI does reach human level (a long time from now), we may be no more capable of understanding its motivations
(1) You're implying they/it'll have "motivations" in the first place. My whole argument was about a lack of motives.
(2) There's a difference between understanding how a conclusion was reached and why it was reached. Example: You may not understand how your compiler decided that line X in your program is dead code, but you may still be capable of comprehending that it in fact is dead code.
The problem is that we don't have that transparency now, and aren't particularly likely to in the future. [1] Particularly for software-as-a-service: I can't audit Facebook's algorithms, despite their effects on what my friends and family see and discuss. And I can't easily audit the black box models used to make public policy, despite the value of transparency there, either.
[1] http://www.theatlantic.com/technology/archive/2015/01/the-ca...
Scale that up in complexity a few orders of magnitude, and the thing may as well be a human. That is, you'll likely have broad understanding of its "motives", but we have that for humans today and it doesn't help that much. I understand Facebook wants to sell advertising, but that transparency isn't sufficient for understanding why a particular message is delivered to your followers while another isn't.
Instead you'll hear "sorry, but it's the algorithm. I understand you need your paycheck, but there's nothing I can do and the programmer won't be available until the middle the of next week".
I learn so much from fiction.
I hope they don't like paperclips.
Surprisingly precise description of life (up to sillicon / carbon substitution)
Crappy classification or regression algorithms being misapplied as cheap, stupid business ideas... are basically nothing like super-intelligent paper-clip maximizers.
In some way it's ironic that with all that effort we've spent on AI so far, simple evolution may yet beat us to the punch as far as AGI is concerned. Makes you wonder about how irrelevant humans are in the grand scheme of things.
The bigger problem IMHO is that spending too much time on those things (and perhaps here, or the Internet in general) leads to dopamine tolerance, aka "ADD". Which cannot be good for lasting relationships.
I secretly believe this will ALSO lead to realizing that you can't "automate all the things," and that some small subset of things will persistently, stubbornly, refuse to be "algorithm'd out".
For the record, if you are a materialist/determinist (Dan Dennettist), then you're basically also agreeing to the postulate that eventually ALL the things (including ourselves) will be "algorithm'd out".
Is this particular usage really dystopian? In the past, probably even 30 years ago, your family would likely have discouraged you from meeting people who were not appropriate to your status.