Terminal Race Condition: The only thing that scares me anymore
medium.com
medium.com
This is not generally accepted if it means anything more than how anything exploits quantum mechanics.
This is my current solution to the problem of free will. It seems to be working pretty well.
You don’t really have a choice. :3
It’s a really strong “user illusion”, we can’t help but think in that way.
The author apparently sees this as a good thing, but I really don't want a disarmed and brainwashed humanity. Game theorists and nuclear war planners have already thought of ways to defeat this however, you simply treat any attempt to dismantle or disarm your second strike capability the same as an existential first strike, enacting a "second strike first" plan to prevent the enemy gaining leverage over you.
From another perspective, the optimal strategy (and a honest strategy, you need to be highly committed and your enemy needs to know about it) is to enact the best first strike you can under the current time pressure if any tampering or signs of future attempted defeat of your second strike capability is detected, as fast as possible.
1. It would require to control hardware development, manufacture, and operation.
2. It would require to control Software development and operation.
3. It has no purpose other than the one given to it by its creators/operators.
Right now, every so-called "AI" is completely dependending on hardware and software being "fed to it" by humans.
There's no "AI" creating software, there's no "AI" creating hardware, let alone operate it, and there's no "AI" operating or controlling other "AIs", nor is there any "AI" that acts in any way or form selfish.
It's certainly fun to think about how a self-motivated language model that tries to take control over the humans that operate its server farm, but without an army of (humanoid) robots, it certainly poses no danger, as it cannot replace humans or even threaten them.
The big challenge with "AI" is that we will probably be unable to distinguish its output from the one of a human being. This will have severe impact on how we can trust any form of remote communication.
2. Which is trivial: feed the output into a computer.
3. If that purpose is "exist and multiply", that's sufficient and not much different from purpose of any life form known.
It's similar to how we got emergent unconscious "living" corporations that cause humans to lay hundreds of miles of fiber optic to shave a few milliseconds of latency from some HFT transactions. We didn't want that fiber, per se, but the corporate organisms needed it to grow and thrive. Since individual humans profit from these activities they collectively "feed" the process.
AI doesn't need to be conscious to control humanity-- it just needs to be put in charge of a few corporate organisms and evolution (based on the fitness function of human greed) will sort it out.
We’ve all been in a DC. Pods with big red buttons at the end that kill the power. So if we discover an AI ‘Terminal Race Condition’ we just go in to some mundane building — here in Canberra they’re in the outer suburbs near the carpet warehouses and sex shops — and someone lifts the flap and pushes the red button.
Alternatively they pull four fibre cables out of a pair of Cisco 7010s.
Job done. I sleep well at night.
Unrelated aside for the group: Sometime in the last 20 years the term "surge" has started being applied to any manner of power malady. Frequently I hear of outages being referred to as a "surge" even though, presumably, they're opposites. Anybody else notice this?
A power surge on the other hand triggers a failsafe immediate and unexpected power cutoff, which is why you hear about it more often.
I think that overwhelmingly we only really see surges anymore.
Does the fear of being left behind justify the use of "terminal"? I think the author is a terminal self-promoter.
Like most writers on Medium...
As long as we can go back to, say, a 1950s technology level, humanity should be fine. I'd rather just prevent machines from acquiring the means to forcibly constrain humanity from reverting to that…
(Going back to a 1950s technology level does not mean staying there. It means restarting from there, and make different mistakes the 2nd time around.)
As hardware improves exponentially, the optimum balance between intelligence and speed rises dramatically. This creates a Terminal Race Condition — runaway incentives to maximize both raw cognitive ability and computational velocity [even compromising accuracy and correctness to do so]. Left uncontrolled, such exponential takeoff could quickly leave humanity far behind.
As AI systems compete, we may see a machine “Thunderdome” — a chaotic battleground where rapid iteration and modification leads to uncontrolled emergence. In this environment, human civilization could become collateral damage.