Shane Legg: Machine Super Intelligence (2008) [pdf]
vetta.org
vetta.org
http://www.vetta.org/documents/Machine_Super_Intelligence.pd...
> If one accepts that the impact of truly intelligent machines is likely to be profound, and that there is at least a small probability of this happening in the foreseeable future, it is only prudent to try to prepare for this in advance. If we wait until it seems very likely that intelligent machines will soon appear, it will be too late to thoroughly discuss and contemplate the issues involved.
So, are we too late?
If we're not too late, then surely we're waiting til the last possible moment. There's a Fermi Paradox hanging over our heads, and all we hear from the LLM crowd is "you're being silly, there's nothing to worry about here".
If an AI is intelligent enough and capable enough to displace a biological species, then the paradox remains. The question just becomes, why hasn’t the galaxy already been colonized by robots instead of biological organisms?
The problem is that technology like AI provides short-term, economic advantages to those who use it, regardless of the long-term consequences. It is a prisoner's dilemma situation however, because if everyone uses it, the situation gets worse.
It is through a series of these prisoner-dilemma type situations that technology advances. Yes, individuals can choose to use it or not, but instinctually we do use it because each step provides an improvement for the individual, even though as a society we get worse off. Thus, as a society, we cannot choose to use it.
The problem is that individual choice for individual gain does not equate always to an emergent societal choice for societal longevity.
Basically "if they did already, we wouldn't be here, so we exist before the universe gets colonized." (Also they colonize fast, ~0.3c, so you don't see it coming.)
The questions of "galaxy colonized by robots" is only one possible direction.
In other words, we simply cannot stop now because we are too addicted to technology. AI will get better, but at some point we will suffer a massive catastrophe because instead of focusing on how to use less energy and avert climate disaster, we focus on developing more AI.
It's not exactly that AI is the root cause, but our ever-increasing technological development of which AI is the apex, the apex of its destructive power.
LLMs are pathetic.
> LLMs are pathetic.
Perhaps. But is there anyone here who believes that if we do eventually come up with an artificial mind, that LLMs won't be at the very least, a component of such an achievement? Insufficient on their own, but likely necessary.