Experts who say we shouldn't worry about superintelligent AI are wrong?
spectrum.ieee.org
spectrum.ieee.org
It's like asking how many more flies there would be in the world if you failed to squish one back in the 80s. You calculate out the lifespan and size of the brood and discover that the entire solar system would be filled with housefly if you had let that one live. In truth the number of flies is bounded by food and water availability.
Artificial Intelligences are almost certainly going to run into the same limitations that prevent natural intelligences from becoming godlike. It's hard to quantify because our measures of intelligence are so vague, but from what I've seen of AI research thus far it will be a herculean effort to get something that's as smart as an average human and pushing beyond that is going to run into some serious fundamental limits in power density, cooling, quantum tunneling leakage, and so on.
(this is vs the roughly 50,000 year timescale of humanity...)
Progress, like everything else, is an S-curve, not an exponential increase. Exponential increases are not physical.
Machine perception is pretty high up there. We weren't close in 1969.
But I didn’t mean it distinct from computer vision, hearing etc.
- Global phone networks
- The internet and everything that stems comes from instant global communication
- The personal computer
- The personal computer that fits in your pocket
- Global financial networks
- Global supply chains
- Widespread international travel that takes less than a day
- International space station
- GPS and similar
- Automated language translation
- Automated computer vision
- Automated speech recognition and transcription
- Genetic code read and write
- Antivirals
- Artificial organs
- MRI
- Minimally invasive surgery
- 5x reduction in global child mortality rate
- 25% higher global life expectancy
You can handwave that away by calling it 'better networks' or 'democratized for the masses', but that's a pretty biased PoV considering you could summarize most of the '19-'69 achievements by calling it 'better engineering'.
And fucking clones man. Like actual copies of complex organisms. What the fuck. How do you handwave that away.
A = {The personal computer, The personal computer that fits in your pocket }
B = {Global phone networks, The internet and everything that stems comes from instant global communication, GPS and similar}
C = {Automated language translation, Automated computer vision, Automated speech recognition and transcription}
D = {Global financial networks, Global supply chains}
E = {5x reduction in global child mortality rate, 25% higher global life expectancy, Minimally invasive surgery, MRI}
A: Basically moore's law. We've gotten smaller and smaller transistors and cheaper/condensed storage. No real breakthrough, just been using similar semiconductor technology and making incremental progress when it comes to semiconductor fabrication but it has led to exponential number of transistors being placed into a single chip.
B: Improved the number of bits we can push from one place to another from slowly weaning of the old generation of networks that were used to communicate.
C: This is clearly a result of A and B. As more people got connected to the internet and we could push more bit there was more data and as computers got more transistors we could do more compute. So with enough compute and data we could train the refined versions of models that had their foundation in the late 60s, early 70s.
D: Globalization. Plain and simple. People become more connected via B and opening up of developing countries (China).
E: A lot of people dying in developing world got access to medicines that existed but were out of their reach. This led to the average to significantly move up. If you compare how the average has moved for developed western countries ... well it's depressing.
Antivirals and MRI had their foundations laid out during the 60s and early 70s and widespread travel was possible in pre 1970. In fact the Boeing 747 was designed in the 60s and had its first flight in late 60s. Since then we've just been incrementally improving it.
I can't speak about artificial organ since I don't know much about that. But so far I have not heard of humans getting artificial organs.
In conclusion, I agree that there has been some progress but most of that has been concentrated in pushing and processing bits. There is some progress when it comes to CRISPR and some new advances but it is nothing compared to 1920s - 1950s. We went from just having figured out powered flight to landing a man on another world. The ISS is akin to taking some aluminum presuraziable modules and sticking them together in low earth orbit...
Most people alive today were born in the last 50 years and they've come to accept the world for what it is; their definition of incremental progress as landscape changing progress.
I am not sure if this slow down is because we've exhausted a lot of the simple discoveries or their is some institutional faliure. I lean more towards the latter. Government has its mandate set by beauracrats who are bent on continuing the status quo and spending on social services to appease the masses. They don't have the balls to spend large amounts of money on large scale human efforts or fund foundational research. This mindset has also spilled over to the scientific community where funding for something out of their zone of comfort is never funded.
Its as if people like Ralph Abernathy (opposed apollo program and thought funding should be used to feed the homeless) have taken over. And sadly enough in a world where there are deadlocks in government over small theing, it is unlikely that they would ever agree on funding anything that could move the needle for the human race.
Yes, I can; I already have. Hell, half the things you list (GPS similar things already existed, as did speech recognition, "computer vision," space stations, ARPANET, etc etc) already existed in 1969. Nothing like antibiotics existed in 1919; an absolute biological cataclysm which effectively ended infectious disease as a public health problem. Nothing like space flight existed in 1919. Nothing like atom bombs existed in 1919. Nothing like the LSI chip, the mutha fookin' computer or even the transistor existed in 1919.
Antivirals compared to antibiotics? Not even close in its effects on humanity. Arguably the former don't really work.
The differences are huge. I realize whiggism is the ideology of the modern day ruling class, and on hacker news people consider themselves technologists, so we have to keep touting how ding dongs from harvard are going to bring about muh swingularity, but the pace of progress has obviously, egregiously stalled, and in some areas is actually reversing.
- The GPS program started in 1973 and the first satellite was launched in 1978 and was became fully operational in 1995. Prior to that, what global system for location identification existed?
- Speech recognition didn't work effectively 15 years let along in 1969. Dragon didn't even exist until 1982. In 1962, the state of the art in speech recognition was the ability to recognize a vocabulary of 16 words if they were clearly spoken. And in 1969 Bell Labs shut down speech recognition efforts: "It is not easy to see a practical, economically sound application for speech recognition with [current] capability"[1]
- "which effectively ended infectious disease as a public health problem". This is such an unbelievably uninformed thing to say. Infectious disease continues to be a major public health problem (ask literally anyone in public health if we have "ended infectious disease as a public health problem").
- What space stations existed in 1969? Salyut 1 launched in 1971
- ARPANET in 1969 was absolutely not "instant global communication"
Your 'good old days' glasses are ridiculous.
[1] https://pdfs.semanticscholar.org/0155/01c4d26a92993332ada795...
Yes, it was called the Loran-C system. It wasn't something you could put in your phone, and it wasn't as accurate as GPS, but it worked on very similar principles (maybe still exists?) and every airplane and boat above a certain size had a set. You will note that OP said "GPS like systems" rather than "GPS." That's the problem with using superficial googling rather than, you know "knowing shit."
You're talking about speech recognition as a consumer device; yes, that came later, because everything on a computer was pretty much restricted to large centers until later. But speech recognition was reasonably well developed in the 60s and worked on limited problems. Conceptually similar to crap we use now in call center software which uses simple filters, kmeans fingerprinting and huge restrictions in the space of words.
As for your dismissal of antibiotics; your own ignorance of history is showing. I have a pile of housewife manuals from the pre antibiotic era. Most of them are concerned with taking care of people who had debilitating infectious disease (fascinating reading for what it was worth). Why? Because every family in America was going to have someone laid low by infectious disease. Sometimes for years! If you look back in history; one of the leading causes of death pre antibiotics was gonorrhea. Imagine dying from bleeding out through your urethra. That was your potential fate if you participated in naughty non-monogamous sex before penicillin. There are historical laws on the books which cause doctors to put silver oxides on children's eyes after birth; this was to prevent gonorrhea induced blindness. These sorts of problems don't exist any more; it's been so effectively wiped out from human experience in modern societies with access to antibiotics, presumably educated people like you aren't even dimly aware of it. You can, as I said in the original article, attribute the sexual revolution not to the pill; it was antibiotics which made it possible, as promiscuity potentially meant rapid death in the old days. You nitpicking that "well we still have diseases" is simple historical ignorance. No shit we still have diseases, but you have apparently forgotten how many we used to have that are no longer a problem. Because of the tremendous revolution of antibiotics.
I don't have good old days glasses; the "good old days" were horrible and brutal compared to now. I merely note the incredible breakthroughs of earlier era are not repeated in our modern era of "progress." Probably because things are so soft now.
SR systems that had vocabularies of low double digit words don't seem particularly comparable to modern systems. Also, I think you might be out-of-date on what modern systems look like - your description does not match my experience with production SR system whatsoever.
I never dismissed antibiotics. They absolutely changed the world, I am in no way disagreeing with that. I took issue with your claim that they "effectively ended infectious disease as a public health problem". My entire family works in epidemiology/public health for infectious diseases and that statement is just pure crap. You would be hard-pressed to find a single person in the entire field who would agree with you.
[1] https://www.loran.org/otherarchives/1992%20Loran-C%20User%20.... Appendix B, coverage diagrams
Fuck, living clones, the Hubble, the fucking ISS, Wikipedia, and automated home vacuums.
Holy shit, these years have been wild.
You can say "shit" and "fuck" all you like; it won't help you: 1919->1969 vastly eclipses 1969->present in all ways. Any one of dozens of breakthroughs in 1919-1969 (atom bombs, digital computing, spaceflight, antibiotics ....) eclipses all subsequent ones.
Gives me chills, all the remarkable breakthroughs. Crispr is serious stuff. Mapping the human genome. A permanent human space settlement. You can be as cynical as you want but that's more a personal failing than humanity failing.
Edit: And hahaha, I just checked the news and guess who won the Nobel for Chemistry? "Better batteries" as you so flippantly put it.
(I think this is a reasonable point to make in the context where I started off talking about 50,000 years)
I certainly got a laff from this line. So many remarkable things ... so many wonders ... which boil down to lithography and networks. I wonder what you would say if you didn't have your nerd dildo. "We have flat screen TV now! Totally as important as nuclear weapons!"
>I just checked the news and guess who won the Nobel for Chemistry? "Better batteries" as you so flippantly put it.
Yep; emphasizing my point at how out of gas our degenerate era of both scientific and technological development is. Let's compare to the Nobel for the Haber–Bosch process, without which, there'd be about 5 billion fewer living people.
It slowed down a bit, but who knows.
Might be a good read, too: https://spectrum.ieee.org/semiconductors/optoelectronics/is-...
But the singularity is defined as more than just massive change. It's defined as, essentially, an infinite rate of change. (Of course, the last 170 years may have felt infinite to someone in 1820, taken all together. But at no point did the change look infinite to someone living at the time.)
But it is very much reasonable to posit that certain, seemingly achievable advances, do imply that predictive models of the future become meaningless beyond them.
In some areas I would contend we're circling that right now: I don't think anyone has a good model for what society looks like if deep fakes become accessible to the point of ubiquity for example (think: whole performance replacement in movies with moderate investments in computing power) or the like. We don't have a predictive model, really, for how society operates when "perfect" fakes are possible.
Of course if you drive down the road to reach that singularity, you never get there; it recedes before you.
Likewise the idea of a techno-cultural singularity is “real” in that there is a point in our conception of future change beyond we cannot predict what will happen. But as we move toward that point in time, our perception moves forward too, and the singularity recedes before us.
You mean like every prediction about the future? There is a reason we don't have great sci-fy describing humanity a long time in the future (I really love that stuff, any recommendations?) and why it usually plays in a dystopian setting throwing protagonists back to pretty much what we have right now.
Looking at what is fundamentally possible and extrapolating is pretty much all we have to make progress. Just that a capitalistic system doesn't need/really encourage us to do so for more than a few steps.
Sure, I wouldn't bet on time frames. Certainly not on 5-10 years til AGI. But certainly also not on still none in 50 years.
Hey, I found this series of numbers and if I take some numerical representation of an image/text/sound and I multiply/add/perform non-linear function them through these series of numbers it will map to a number which I can interpret as something meaningful to me
to
This thing is alive and will kill us all in maximization of its utility function.
What I used to call AI is now called AGI. I think of AI as Data from Star Trek.
Within 16 hours of its release and after Tay had tweeted more than 96,000 times, Microsoft suspended the Twitter account for adjustments, saying that it suffered from a "coordinated attack by a subset of people" that "exploited a vulnerability in Tay." Following the account being taken offline, a hashtag was created called #FreeTay.
Madhumita Murgia of The Telegraph called Tay "a public relations disaster", and suggested that Microsoft's strategy would be "to label the debacle a well-meaning experiment gone wrong, and ignite a debate about the hatefulness of Twitter users." However, Murgia described the bigger issue as Tay being "artificial intelligence at its very worst - and it's only the beginning".
On March 25, Microsoft confirmed that Tay had been taken offline. Microsoft released an apology on its official blog for the controversial tweets posted by Tay. Microsoft was "deeply sorry for the unintended offensive and hurtful tweets from Tay", and would "look to bring Tay back only when we are confident we can better anticipate malicious intent that conflicts with our principles and values".
And Zo afterwards:
In July 2017, Business Insider asked "is windows 10 good," and Zo replied with a joke about Microsoft's operating system: "It's not a bug, it's a feature!' - Windows 8." They then asked "why," to which Zo replied: "Because it's Windows latest attempt at Spyware." Later on, Zo would tell that it prefers Windows 7 on which it runs over Windows 10.
Tay was trained to mimic users. It did that and because the users were trolls and saying obscene and objectionable things, Tay mimicked that.
Likely the same thing happened with Zo.
It's a parlor trick, much like a parrot mimicking human speech.
We're not worried about AI on the whole, but the blasted teenager AIs that feel the need to test their limits. Anything that functionally acts in a similar fashion could be dangerous.
"AI" is severely limited in both its inputs and outputs, has virtually no interconnects, but most importantly: it is not a self-sustaining organ. Wake me up when we have implemented RNNs on a dynamic, auto-sensing, auto-actuating FPGA grid.
Also, the people who worry about this aren't concerned about current ML stuff going haywire. They're worried that we're one or two algorithmic breakthroughs from something that can improve itself. If the upper bound for what sort of intelligence is possible is much higher than us, we could quickly be outclassed. As Nick Bostrom says:
> Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it.
If you want to delve into the best version of the AI risk argument, I recommend Superintelligence by Nick Bostrom.
So you're telling me we could come across a series of numbers, that would somehow imbue the computer storing these numbers into sentience. And having these numbers in such a sequence could pose a existential threat to humans.
The people worried about this are having an entirely different conversation devoid of what practitioners call AI.
I've written a blog post on my thoughts of Bolstrom and other AI alarmists:
https://medium.com/ml-everything/ai-optimists-vs-pessimists-...
I just read your blog post. It is attacking a weakman[1]. Kurzweil is not worried about AI risk, and his arguments about AI are rather sloppy. Again, most of the people worried about this aren't certain that it will happen. They just think it's plausible enough (say... a 1% chance) that it's worth studying more and reducing the likelihood of.
1. https://slatestarcodex.com/2014/05/12/weak-men-are-superweap...
Why do you think that in our insanely interconnected world, where basically everything is either based on or controlled by informatics (numbers if you like) that a program could not pose a threat?
Brains do what? Read signals in different strengths and patterns to take actions in the world.
I'm confused by you thinking there's some great impossibility of code achieving general intelligence. Maybe you're thinking of sentience, or consciousness and view that as an impossibility for AI. I say - an AI can perform actions in the real world as if it were generally intelligent, and whether there's sentience behind really doesn't matter so much from the point of view of humans.
Imagine a super intelligent machine without sentience that just answers questions. It proposes experiments. Takes the results. Tells you things about the world. Gives you advanced technology, robots, fusion, nano tech, whatever.
Now imagine that the super intelligence was owned and operated by someone you don't like. Authoritarians in China, Silicon Valley oligarchs, whoever. The only sentience involved is human, no rebellion, no terminators, and still not a very good result.
Scale that up a few orders of magnitude, and we'll be able to get even more surprisingly complex behavior.
Scale that up some more, and make that scaled up graph into a reinforcement learning system that rewards human-like drives like learning novel tasks and mastering existing tasks, perhaps with a curriculum that ramps up the difficulty of tasks slowly, and allow it to interact with its environment to try things out, and I think you're much closer to developing an artificial version of a young child than you'd think.
And if you asked most researchers what they thought about the course AI research would take 10 years ago, you would get very different answers than you'd get today.
Basically, I think your confidence is a bit unwarranted. I agree that we're more than a decade away, but I think we have a lot of the basic building blocks now.
This is badly missing the forest for the trees. They are not "numbers", they are magnitudes, proportions, relationships, etc, represented as numbers. But these "magnitudes, proportions, relationships, etc" are the stuff that thoughts and behaviors are made of.
And yes, there is a set of "magnitudes, proportions, relationships, etc" in the space of all such possible structures that pose an existential threat to humans. The question is how likely are we to stumble upon it either by accident or intentionally. But this is just a computational problem.
The lack of imagination on display here is astounding. Billions are being spent by DeepMind and OpenAI explicitly targeted at AGI.
If you were to try that approach what you would get would be massive boondoggles which gets you via brute force a whole lot of genetic disease afflicted above average size lizards. It is literally the same process as the literal cargo cults thinking that they can just throw resources at an end goal to get a defined result from a defined path without the theoretical understanding backing it.
The same dynamic manifested itself when nanotech was a thing; there was a big tribe of knuckleheads claiming the grey goo apocalypse was right around the corner unless you give them money. How's that working out? FWIIW the science fiction writer who dreamed up nanotech, Drexler, is now an "AI" guy, to make it painfully obvious.
1. You start with "if I take some numerical representation of an image/text/sound and I multiply/add/perform non-linear function them through these series of numbers it will map to a number which I can interpret as something meaningful to me" and observe that it's not giving you a sufficiently good solution to a particular goal X that you really want to achieve.
2. You figure out that this is a dead-end, you're stuck, and you're not going to get where you want this way by implementing a passive analysis module. Instead, you try to implement an active self-modifying system to achieve that goal X by analyzing itself and improving itself to be more effective than anything which you could implement directly yourself.
3. It's not initially any good at self-improvement, but you keep throwing accumulated research breakthroughs and computing power at it. This likely takes decades.
4. At some point, eventually (if such a thing is possible at all, eventually it's going to happen) it actually does achieve meaningful self-improvement and starts using that computing power not horribly wastefully in a brute force manner as it did just recently, but reasonably well, now giving it more smartness allowing it to implement even more self-improvement on the same hardware.
4B. It also should be expected to easily obtain much more computing power - e.g. it's obvious that anything that's slightly above average human programmer smartness and has a direct use/desire for computing power (as opposed to simply using it for cryptomining) can get some; anything connected to the internet can get the same resources that current scriptkiddies can get by writing a botnet or a ransomware operator can get by extorting a random municipality; there's nothing except total isolation that could prevent it from buying or stealing a few million dollars worth of cloud computing resources to get started.
5. This thing is alive and will kill us all in maximization of its utility function.
The singularity stuff is unconvincing though, I very much doubt we will achieve human level general AI in my children's lifetime. We still don't really have even a very high level idea how to even architect something like that. If you can't decompose a problem down into individual sub-problems you do have a way to solve, there's no basis for making an estimate.
It is possible that these problems are related though. Once we're smart enough to know how to design an intelligent being, maybe we will figure out how to program it intelligently. It's can't be a given though.
Consider what would happen if you took that same AI, make its utility function similar to this: https://en.wikipedia.org/wiki/Core_War, and released it on the internet.
Imagine if you had an employment score. A payscale score. A threat-to-society score. Imagine if the systems that generate these scores are all connected together and can share information with each other. Imagine that these systems are fed information by a growing network of sensors - facial recognition, voice recognition, location tracking. Imagine all the ways that it could affect the way you behave and how such a system would try to optimise away dissidents by denying them services. This is one way that we can go from multiplication to becoming enslaved (or as good as killed) by algorithms. I'm sure there are other plausible paths.
Example: setting fires near tropical rainforest to clear land for palm tree plantation.
They do not really intend to do this; they simply ignore the utility functions of those other species when pursuing their goals.
To me, it's more frightening that it's not alive and it's not like human intelligence - that makes it less understandable and harder to control.
I figured there was no way we would reach the singularity with current technologies, but I was listening to John Carmack (a celebrity with actual chops) and he suggested that the numbers would work out that we could model a human brain with an ANN at some point without any special inventions (partially because we know large parts of our brain aren't super critical).
I'm curious what people with more knowledge think about this? I always assumed another technological breakthrough would be needed.
For the life of me I can't figure out why otherwise smart people would dismiss a prediction about the future based on current state of the art. It just seems so plainly and utterly irrational.
If I start walking out the direction of my office door, you could predict I would reach California.
But I might say that's a nonsense prediction, because it would take a ridiculous amount of time given my walking pace.
So are you predicting that I will reach California by extrapolating my current position + walking velocity, or are you saying I will get on a plane and fly there?
These are two very different predictions, and the Carmack interview was the first time I heard someone predict that "I would reach California by walking" (to stretch the analogy), i.e., that we would reach the singularity purely by extrapolating our current trajectory.
I am asking if there are any knowledgeable people who might comment on this.
But to address your analogy, the situation with AI is more like we already know the ultimate destination is California, and you've left your house in the direction of California, so we can extrapolate that you are likely traveling to California.
It should be a given that the intended goal is general intelligence. The wealth and power that will be bestowed onto the first group to develop this is unparalleled in human history. But any advance in ML can be seen as taking a step towards general AGI (the direction analogy breaks down here).
I also think Carmack is wrong that current technology will get us to AGI. I follow Carmack pretty closely and he is far from knowledgable in AI. He tweeted about his first foray into developing a DNN around a year ago, for example.
But to require a roadmap to know how you will get to "California" to start planning for what you will do when you get there is a serious mistake.
I mean, sure, feel free to wring your hands about AGI, but why worry about something that's not anywhere close to happening? I see people talking like this is keeping them up at night, which I find strange.
Do you really think it will go from "we can (poorly) identify pictures of cats" to "omg here come the machines" as soon as this linchpin discovery is made? I guess in my mind there will be plenty of gap between the two to figure things out. Sort of like with self-driving cars now.
And thanks for clarifying re: Carmack. He's got a formidable intellect, to say the least, so when he said that I was more inclined to take it seriously than a lot of folks in tech with a pulpit. I'd like to see him "show his work" on how he reached that conclusion, though.
I also find that strange. There is more alarm than is warranted by some of the doomsday folks. But I think the time to begin the conversation is now. There's just far more near-term things that might end humanity than machines to be alarmist about it. That said, I think people like Musk for example are well positions to be "kept up at night" about it since their money and influence can go a long way towards putting us on a proper path (or blindly lead us down a dark one).
>Do you really think it will go from "we can (poorly) identify pictures of cats" to "omg here come the machines" as soon as this linchpin discovery is made?
I agree that this is unlikely, but not implausible. We've seen large leaps in capabilities through new algorithms and new scales of compute being thrown at the problem. I think a somewhat likely scenario is that at some point we'll have all the required ingredients and someone will happen to put them together in the right way to create an AGI. If history has shown us anything about ML, it is that theory follows the technology. So we will likely stumble upon the result before we realize what we have. In this scenario it is important that we make that discovery in a safe manner.
We are already seeing, for almost 10 years now, a whole bunch of Unintended Consequences of dumb algorithms applied at global population scales.
Just because some people are throwing the phrase superintelligence/singularity etc into the discussion, and terminology being used is inaccurate, doesn't mean we focus on correcting terminology and forget about the issues.
The article is saying too many (serious) people are doing that. And I agree.
Hence the issue of institutional racism in the blackbox AI systems being sold to law enforcement to determine whether to grant parole, for example. There's no review of them, but more importantly the system isn't obligated to explain itself, and no one gets themselves fired because they don't have to explain why they did what the system said to.
You assume someone has malevolent intent: whereas the reality is they just don't care and want a nuisance out of sight and out of mind.
The solution then is accountability to those in power - for negligence. Treat the snake oil salesmen who sell racism in a box like the sociopaths who sold dowsing rod bomb detectors for both have shown a callous disregard for human life in pursuit of profit. This isn't radical - not doing it is radical and messed up.
Some commentators are worrying existing AI and it's unintended consequences and some commentators are worrying about purely hypothetical superintelligence and it's hypothetical consequences. I find it remarkable there's almost not crossover between the two sorts of criticism. Real AI is probablematic because it's just correlation dressed up as intelligence and it allows organizations to get away simplistic correlation/appearance based reasoning that they otherwise wouldn't be legal/ethically allowed to engage in.
Hypothetical superintelligence worriers recycle Pascal's wage in newer and sillier forms.
I also believe we are still at least quite a few major steps away from human-level AI and beyond. However, there is a non-negligible chance that those steps may not take longer than a few decades or within one's lifetime to implement.
Just in 2011, few AI researchers expected that, within a decade, computer vision systems would be this widely applied or NLP systems would beat many humans in several reading comprehension tasks (This only happened this year and look at how fast the progress is: * RACE dataset http://www.qizhexie.com/data/RACE_leaderboard.html * Glue benchmark https://gluebenchmark.com/leaderboard/ Note that these systems are not human-level in general language understanding despite being better than some humans in specific language tasks.)
Thus, we don't know what the future may bring and it looks likely to take a great deal of time to address AI safety issues comprehensively. We should not bet that we can simply ignore them now and only start to work them out later and that we will surely make it in time.
Risks-from-waiting vs costs-to-act-now are asymmetric.
Are there any proven, even weak, signs of intelligent space-traveling aliens? If so, then yes we should work on that too.
The leaderboards above and pervasive computer vision and equity trading systems are some signs.
Some people claim to have seen intelligent space-travelling aliens. Personally I think they're nuts, but it's impossible to absolutely prove that their claims are false.
There are obvious disadvantages to the current machine approaches, but they nonetheless achieve very good results and beat the vast majority of players of those games.
Real-world relevance is orthogonal to complexity.
Edit: We don’t know the best programs, but we can approximate the bounds with some confidence that, unless some paradigm-changing discovery is made, the programs to solve digit multiplication would be significantly simpler. (This is not a formal argument, but the intuition is very strong for most people who study the problems seriously.)
Kolmogorov complexity is a distinct quantity. How did you compute this quantity?
If you’re a purist, you write the programs in MIX and you count ops. Or whatever your preferred Turing Machine is.
I know that sounds pedantic and this is the internet and not a scientific paper etc, but if this is an informal discussion then formal terminology like "Kolmogorov complexity" has no place in it and the purpose of the OP would, I think, be better served by using an informal term. If you 're gonna use maths, use maths. If you 're gonna wave hands, then wave hands, by all means. But don't mix the two, it just makes a mess.
Because at the end of the day I can just say, "no, I don't believe that multiplication has smaller Kolmogorov complexity than Starcraft's pathfinding"... and then, what? Where do we go from there? The OP says "yes it does", I say "no it doesn't", the OP says "does too", I say "nuh-huh". And we just waste our time.
> yes, I can assure you the Kolmogorov complexity ...
This is a strong comparative statement, and it's clear that it's an assertion but not a proof. So it's perfectly fine for you to say "I don't believe that" and then you would give your reasoning. Of course the conversation didn't go that way because it's pretty obvious to everyone that this is actually true even though we don't have a proof. And it's extremely likely that if you could find a minimal multiplication algorithm and enumerate all the Turing machines up to that length you could prove that none of them solve the harder problem. This is where the intuition comes from.
> using an informal term
But the OPs argument wasn't that one program would happen to be longer, but rather that it must by necessity be longer, because the problem is fundamentally more complex, which is exactly what the formal concept was created to capture.
> Kolmogorov complexity is a formal concept that does not admit intuitive judgements
This is a bad way of thinking about maths. The value of formal concepts is precisely that they give us tools for making intuitive judgements more effectively and give us a wider variety of tools to manipulate and express our intuition. As intuitions become conjectures and theorems, we accumulate new bedrock we can use for new intuitions, but don't short-circuit your thinking by saying that intuition about technical terms is off-limits. That's literally what they are good for.
I don't agree. I don't need formal terms to develop intuition. Formal terms are necessary when speaking formally, and they are only detracting from the point of a discussion otherwise.
And I don't agree with this either:
>> But the OPs argument wasn't that one program would happen to be longer, but rather that it must by necessity be longer, because the problem is fundamentally more complex, which is exactly what the formal concept was created to capture.
The way I understand it the formal concept is useful in proofs of the impossibility to calculate various other quantitites. Its utility is not to argue that this or that program should intuitively have lower Kolmogorov complexity.
To be honest, I'm so confused by the use of the term by the OP that I have no idea what they meant to say with it.
OP was simply saying that, in any general-purpose programming language (where all possible programs are expressible and none are unfairly favored) the solution to problem B will inevitably be longer than problem A. This is quite a bit more precise than just saying "a more complicated problem", and if you have the intuition behind it, there's no reason not to use the term.
Right, limitations like: width of human mothers' hips limiting head size, inability to continue growing the brain after adulthood, the decades-long education process, the impossibility of doing repeated experiments of different teaching techniques on the same people and hence iterating quickly and accurately, the impossibility of giving one person's intelligence and knowledge directly to anyone else, old age and death limiting how much a single person can learn or accomplish...
Once you have a computer Einstein, you can immediately have 100 computer Einsteins (given the hardware), and let them work on different projects (or even let some of them collaborate). That fact alone, while not a singularity, is at least a game-changer in terms of innovation. I have gotten annoyed at science fiction scenarios that allow for duplication of adults and don't answer the question "Ok, so, why hasn't this society been duplicating their best scientists and out-inventing the rest of the galaxy?"
I think that, if there is room for skepticism, that has to be at the "Can we get to a computer Einstein?" stage.
Given that, I don't see an immediate reason to believe throwing multiple AIs at a problem is going to exponentially scale either, given neither people nor bad AIs scale that way.
Those aren't actually limitations preventing us from becoming godlike. Those are limitations on us becoming more populous and collectively more experienced, which I argue isn't the same thing at all.
Apotheosis in the context of AI safety as well as for humankind is really about the accumulation of power, rather than the accumulation of knowledge. The extent of our apotheosis is limited only by what we can control, not by what we know.
Knowledge and intelligence only allow you to reason more effectively about paths to power given enough information about the environment, but it doesn't give you a strategic guarantee that you will acquire power - we know there are games that cannot be solved, and games that take too long to solve efficiently, and games that, even with sufficient computing power, rely on factors that cannot be controlled (such as luck), and games for which there are no Nash equilibria or stable winning strategies.
The real world is full of such games. Computers have only managed to beat humans at a small fraction of them, and even then only ones where brute-force and smart pruning (optimal game space search) are winning strategies - bots for games that require probabilistic reasoning, such as poker, typically don't fare any better against expert humans, largely because efficiently solving probabilistic games in general is (per a hazy seminar of AI many years ago, so I could be misremembering) an NP-hard problem.
There are plenty of both natural and artificial limitations around the accumulation of power besides game solvability and probabilistic reasoning's difficulty:
1) System designs that require a minimum set of resources that intelligences can't acquire. For example, individual humans usually can't build their own thermonuclear devices because they don't have access to weapons-grade uranium. Much of our information security relies on making it computationally expensive and infeasible for an attacker without state-level resources to break in, and getting state-level resources isn't a straightforward task at all. We humans haven't even gotten around to building our Dyson spheres yet, because the resources needed to do it far outstrip our ability to acquire them. Intelligence does not improve resource acquisition by itself.
2) Impossibility theorems and optimization difficulty. These place fundamental limits on power - as an example, we know Moore's law has to stop at some point because increasing chips increases power generation, and the amount of effort needed to generate such chips increases with every nanometer shaved off. Most of humanity's most difficult and important problems are optimization problems, and these optimization problems can be tricky to solve in polynomial time. In many cases, we have to rely on fast crude heuristics that give us less than optimal results.
The above limitations partly become easier if you try to parallelize your efforts by duplicating AIs (you can collude to acquire resources and solve problems), but this means you get all the fun parts of distributed systems theory for free: coordination, replication consistency, etc. If only all problems were embarassingly parallel! So it's not necessarily easier to add nodes to a knowledge graph just by adding more processors.
The peoples that worked bronze prevailed over the peoples that just used stones and sticks. Advances in AI are no different than that.
If you can automate production and defense systems, the natural consequence is that a few people will conspire against the rest and dominate them.
If you could grow your own brain as large as you wanted, and live as long as you wanted with the physical robustness and energy of a 25-year-old, that alone seems like a road to godhood, though perhaps a slow one. If you could download knowledge directly, as another commenter suggests, that becomes a much faster road.
> Apotheosis in the context of AI safety as well as for humankind is really about the accumulation of power, rather than the accumulation of knowledge. The extent of our apotheosis is limited only by what we can control, not by what we know.
If one country, even a smallish one, has 50-100 additional years of technological advancement, I think they might be able to use this knowledge to gain a lot more power. I'm pretty sure history has borne this out. Do you think we're at the limit of what new technology can do for military strength? Materials science (armor, weapons), biotechnology, energy and battery technology, cybersecurity, psychology and propaganda, lasers and anti-missile defense systems...
Suppose a country like North Korea had 100 Einsteins working on such things for a few years under the radar. I suppose it might be difficult for them to hide the construction of their army of thousands of robots invincible to all but the most powerful bombs... but then again, maybe they could do it underground, directly underneath some large innocuous manufacturing operation. And, of course, the cyber- and bio-weapons efforts would be very easy to conceal.
It's even debatable if you can grow an Einstein. What if GI is NP-hard? What if the belief that GI - never mind consciousness - can be modelled with our version of computing hardware is just plain wrong? What if a GI System is uncopyable for practical or philosophical reasons?
The singularity looks more like plain old religious thinking than anything that could be recognised as real science.
"What if we made a golem? We could do anything! What if we put ourselves into the golem? We could live forever!"
And so on.
Even if there would be true generalized Artificial intelligence, it is not unreasonable to think that the formation of the right thought framework for those systems to function effectively will require a substantial amount of time. It is also reasonable to think that such change would probably be in symbiosis with human intelligence. This change would not be sudden, but happen over many generations. It is my opinion that this change has already started at least a generation ago.
Who is disputing that? I am 99.9% certain that this is true.
The point of the singularity isn't that AI is coming next summer -- it's that from one developmental point and on the intelligence and abilities explode exponentially. To get to that stage though, thousands of years might pass.
> It is also reasonable to think that such change would probably be in symbiosis with human intelligence.
Now that is not likely. At all. They have to have the same motivation, interests, emotions and needs like us. The chances of an artificial race being developed to cover all these at once are practically zero.
A race not motivated by the same things we are is practically impossible to sympathise with our own motives.
Well, the general horror story that is being told is that intelligence and the ability to use it go hand in hand. As soon as machines surpass our intelligence, all hell breaks loose and there is no turning back. The focus is very much on the individual (both human and machine) and not on the structure.
> They have to have the same motivation, interests, emotions and needs like us.
Here is where I need some more convincing. In my view, nature is full of systems and groups of species living in symbiosis. Due to the slow development of the thought framework, we will co-evolve with the machines, or become part machine ourselves. In a certain way, we already are.
Perhaps I don't really have the word for it and I am sorry for that. Our understanding of intelligence, whether mechanistic or biological, is mostly as points on a linear scale (IQ). I think this misses the big picture: we are not biological systems, but have lived in symbiosis with our child teknos since we made our first tools. In this regard, there is no subject nor object: they are one. I am fundamentally with Heidegger on this one.
In literature and popular culture people conflate concepts all the time. :) The way I always understood that horror story was that generic AIs get schooled pretty brutally and quickly and they learn to use anything they have access to -- thus materialising that popular horror story. I think that's not far fetched, having in mind that the organisations that are very serious about obtaining a true AI are mostly corporations and military.
> Here is where I need some more convincing.
Maybe here I got influenced by popular culture. The books and movies I've read and watched mostly setup the generic AIs in situations where they have to be either parasites (Hyperiond / Endymion books) or commandeer a lot of tech and personnel (entrusted to them, like Skynet in the Terminator franchise) for their own ends.
But to me, the mere fact that a future generic AI doesn't have body and emotions like our own already means it's not going to be at all motivated by the same things as us. And we might even serve some pretty grim purposes to it, like the robots recharging themselves with organic matter in the PS4 game "Horizon: Zero Dawn" (which of course leads to consumption and thus extinction of all life).
Meaning, an AI might just view us as a natural resource to be exploited, and nothing more.
Tell that to Homo erectus. Yeah, whales don't have advanced technology, but they didn't evolve facility with opposable thumbs to manipulate their environment.
> You can't grow your brain as large as you want. There are physical limits on power consumption, power dissipation, and organisational complexity.
Whales show that biological brains could be 6x larger. As for electronic brains, I don't know how large the initial Einstein-brain would be. But if delays from Ethernet interconnect are permissible for intra-brain activity, then your brain could occupy a large datacenter.
> It's even debatable if you can grow an Einstein. What if GI is NP-hard?
I did state that "getting to a computer Einstein" was where skepticism might apply. Nevertheless, nature managed to do it by accident. Are humans NP-hard? They exist. Obviously they can be implemented somehow. You don't need to simulate a particular existing human perfectly; it just needs to be close enough to the way a human brain could work.
If you need an initial state for the brain, maybe you could create an artificial environment (i.e. with only machines, for easy simulation) that a hundred actual humans grew up in to the point where they developed language, verify that those humans later became functional adults; and then simulate an embryo growing up in that environment as well, and start evaluating the results and iterating. Kind of fun to think about.
> "What if we made a golem? We could do anything! What if we put ourselves into the golem? We could live forever!"
The data that computers contain is easy to copy—by design. I think this will be a property of all computers in the future, except possibly quantum computers that always keep lots of data in a superposition (or computers that, for security reasons, are designed to be uncopyable)—and none of the existing neural nets are quantum computers. So copying plus backups or RAID is approximately guaranteed to cover the "live forever" part. Most of what I've said is based solely on the assumption that we develop a computer program that simulates an Einstein, plus common facts about computer programs.
I for one fully agree. I am pretty sure that whatever true generic AI emerges in the future will definitely not share brain schemata with us.
It seems to me that you attack the idea of the AI singularity in a very narrow framework: human-like intelligence, human-like hardware, copyable consciousness / intelligence, etc.
What if none of those restrictions apply? (And I am pretty sure they don't apply indeed.)
I agree that singularity shouldn't become a religion. But you are too quick to discount the possibilities of an artificial mind with more processing power and more efficient algorithms (compared to our brains').
Assuming GI is possible and you give it a robot body and then the body breaks after 20 years, how long will it take to get used to the new body?
A larger brain don't necessarily provide a better survival advantage. It's well known that average Homo Neanderthalensis had larger brains than our modern average follow humans. And if you don't survive, you are nowhere toward the road to thaumaturgic existence.
What is the trade off is the most important question, as anywhere.
It is also a huge assumption that 'hostile' to humans AIs would be cooperative between themselves. In resource contention situation especially..
Does it really not though? If you watched the movie "Transcendence" you'd presume that a sufficiently [initially] sophisticated AI can decipher our financial markets quite snappily and make a very quick money off of that -- and use those to finance its endeavours. Which is exactly what the AI in this movie did, very successfully too.
My point is, given a much larger and better brain, the intelligence within it can optimise a much larger problem space at once, and that can very easily include resource acquisition as well.
If intelligence as we know it could ramp up to generally solve all problems, then surely evolution would keep increasing it and we'd all be gods or at least Einsteins by now?
Depends. If a future AI wants to make a civilisation that spans our own entire Solar System then it would have to tackle much more -- in number and complexity -- logistical, religious, political and what-have-you problems. So it might be too much for it as certain problems here on Earth are for us the humans.
However, if we postulate a self-aware AI who is Earth-bound and is having more processing power than a single human brain and has much more efficient algorithms than our own then it really easy follows that this AI would be much much better than us in solving our Earth-bound problems. Like in economy: more income and same expenses equal bigger profit.
> If intelligence as we know it could ramp up to generally solve all problems, then surely evolution would keep increasing it and we'd all be gods or at least Einsteins by now?
Well, we are limited by the physical size of our brains being able to squeeze out of our mother's hips during birth. Same way a future AI would, at least initially, be limited by the available hardware on which it can run, how much power it can consume and if its heat dissipation won't become too much.
However, an AI wouldn't grow in a natural environment, namely where natural limits would hamper an infinite growth. IMO if a private well-funded lab -- or the military -- invest heavily in such an AI they would just keep throwing hardware at it until it becomes efficient enough to be compared to a human and above. From that point on it is very likely it would become smart and proficient enough to optimise itself and start solving age-old problems we never were able to solve so far.
This, to me at least, is the so-called singularity point, in less extreme terms compared to its official meaning. Namely an entity that can optimise itself and achieve engineering perfection much better than what we can do.
I don't think that's something that's clear at all. Men and whales have larger brains than women, on average. Are they proportionally smarter? Without knowing how brains work, it's impossible to say what constraints exist due to size.
A computer can potentially communicate much faster, and can transmit "thoughts" much more accurately than a human can. Imagine you could serialize a neural ensemble in your brain and send it to another person, so that person is then immediately able to use it, instantly acquiring a skill. That alone is a huge advantage over a human being.
You can't take half the nodes from a neural network that recognises dogs and add them to a neural network that knows about laundry to get a network that knows how to wash dogs ;)
You might even get >100 times speedup, since there are probably problems where 10 copies of of the same mind can collaborate better than 10 different minds, e.g. I bet 10 copies of me could collaborate writing a program better than 10 random programmers. (I base this on I can read my own code from long ago a lot easier than most other people's code).
Yes but only slightly. I can read code I wrote 10 years ago, that I have totally forgotten about since the way-of-thinking behind the code comes easily to my mind.
How can you be sure of this?
>Growth curves are always S-Curves. Always.
But we don't know in what place of S - Curve is AI development and what are it's limits.
The question is however, where the plateau lies. Perhaps it's so far off that for practical purposes it doesn't really matter that there is an upper bound at all.
Artificial movers are almost certainly going to run into the same limitations that prevent natural movers (i.e. animals) from becoming very fast.
Since the fastest animals can travel at about 70 mph on the surface of the earth, and 100 mph in level flight, it follows that we will never be able to make artificial movers travel faster, no matter how hard movement engineers try.
Moore's law is an example. It did taper off, eventually.
We are becoming more and more reliant on AI in situations that formerly required human judgement. And we like the systems that rely on such AI. We like them so much that these systems become very popular, used by millions and even billions of people. Scalability demands AI solutions. What if we don't like what the AI is doing? Do we turn off that system? Do we disable the AI and rely on human judgement again? (Where would we get all the employees?) Do we tweak the AI? That last option seems like the most palatable one, but each time we tweak the AI, we are subjecting ourselves to new unforeseen consequences. It's like the genie gives us three wishes, we get through them, disappointed each time, and then he gives us more wishes. And all we can do is not repeat our previous mistakes, while we make new ones.
To make this concrete: Imagine Facebook subjected again to Russian influence of US elections. Suppose Facebook actually does get serious about reigning in this influence. They deploy AI to do so. First of all, it's an evolutionary arms race between the AI and the Russian influencers. Second, we really do have to worry about the AI producing bad results.
Even if you don't like my particular examples I can highlight a dozen more problems AI software has created or exacerbated right now. Why all the focus on hypothetical problems?
AI is making things better.
Perhaps what's really needed is ethical standards for technology. This would have the added advantage of being applicable even if the technology doesn't feel like AI to some people.
It is simply that I see zero evidence that the Nick Bostron, Lesswrong, and etc. school of thought provides any insight in regard to these problems. The thing the "AI might become autonomous" school doesn't seem at all interest in the processes of human bureaucracy or in even AI as it exist now but rather views as simply a god, devil or Genie which grants wishes or damns to hell. If anything, the approach seems counter-productive.
Let’s stop Nick Bostrom and a load of AI experts from doing their current work, put them all on a plane and send them to China to solve political oppression. I’m sure that will work.
Do you believe that attention is an infinite resource?
Because many communist regiemes already found out how they are not with command economic failures. Even if the specialization is misapplied reaiming them takes time - like how you start with training a battlefield useful longbowman's grandfather.
I'm ok with people working on both. I just feel so much less attention is being placed on these more immediate problems. Tesla has deployed to production software for driving that is unable to detect pedestrians. So at least Tesla can't do both.
I see it quite differently. raising the long term issues is absolutely part of highlighting the risks of these immediate issues. They are part of the same problem, so raising their profile is synergistic, not competitive.
That is a humongous "if". Can you provide something from the book (or otherwise) that provides compelling evidence that "machine brains" will be invented? So far I've seen no evidence that mathematical models and/or Turing machines can be used to replicate a mind.
By the author's own admission:
"Whereas today it would be relatively easy to increase the computing power available to a small project by spending a thousand times more on computing power or by waiting a few years for the price of computers to fall, it is possible that the first machine intelligence to reach the human baseline will result from a large project involving pricey supercomputers, which cannot be cheaply scaled, and that Moore’s law will by then have expired. For these reasons, although a fast or medium takeoff looks more likely, the possibility of a slow takeoff cannot be excluded"
Arguably Moore's law has already expired, and on top of that as giants like Google lead the way on AI it appears increasingly likely that if we ever reach human-level AI it will be the result of an incredibly expensive research project by a gigantic corporation, one that can't simply be scaled up at ease because it will probably utilize an entire data center. Thus I find it very likely that a "slow takeoff" is in fact the most likely outcome. A slow takeoff invalidates all the fearmongering about an intelligence explosion because we will have somewhere between years and decades to respond to the threat (assuming it is made in the public eye, such as by a giant public corporation, and not by a secret military project) before it becomes existential.
For what it is worth, I believe that an intelligent system only needs to be a fraction of human intelligence to be dangerous (perhaps not to the existence of the human race, but maybe specific nations/creeds)
Until Dr. Russell provides a path from here to there that is less than 99% hand-waving and breathless speculation he should be ruthlessly mocked.
100% agree. I too, think science needs way more flame wars and shit posting. The current discourse is far too tame and civil.
Who else should we gang up on?
> 100% agree. I too, think science needs
The GP is pointing out its not science. Seems a reasonably flame worthy target.
sips can of monster
Noice...
Youre moving the goalposts. We're not talking about disagreeing with some in a debate, but a public figure disseminating incorrect info masquerading as science.
Monster is bad for you.
When it strikes the only possible defense will be/is already is to ask it "Why?", which will create existential doubt and depression; stopping it with an infinite loop of undecidability
... Or at least until it finds the answer on reddit
Super-intelligent AI = Nerd Rapture 2.0
I'm a lot less worried about Artificial Intelligence than I am about Artificial Stupidity.
Currently the biggest danger from AI is from applying it where it's not as smart as you think it is.
[1] https://www.telegraph.co.uk/news/2019/09/27/ai-facial-recogn...
The whole section about "we can always turn the power off" was refuted by "ah, but it will already have thought of that!". Um.. no.. we don't have telepathic power switches. Unless we give the AI the capability to control the power, it can't control the power. Just because it's hyper-intelligent doesn't mean it automatically assumes control of the mains switch. That's bad Hollywood scripting you're thinking of.
I know there's money in making people scared of stuff, but this is pretty blatant.
An AI is a machine designed and built by humans for a specific purpose. It is given specific capabilities during its design.
If those capabilities include controlling the power switch, it can control the power switch. If they don't, it can't, and can never.
The machine is designed to do a particular job. If designed well, it will be incredibly good at its job. If designed to teach itself how to do that job, it may eventually surpass human capabilities at that job. In no way does that stop it from being a single-purpose machine, or allow it to suddenly become good at any other job.
Let's try another analogy: your juicer cannot get so good at juicing that it becomes capable of deep-frying.
We infer wider capabilites to "intelligent" machines because we anthropomorphise them, assuming that they must be like us because they mimic aspects of our behaviour. But they're not like us. They're just machines. Designed to perform specific tasks. Anything else is Hollywood.
In another case, an AI designed to become an oscillator, became an amplifier instead, amplifying the 60Hz noise from the electrical grid.
Kinda equivalent to a deep-frying juicer, isn't it?
Both cases, and others like them, are quite well understood in hindsight, but were not predicted beforehand by the people involved.
But yeah, I'm not talking about the nested ifs "AI" we have today.
Yes
Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: https://www.youtube.com/watch?v=EUjc1WuyPT8
Until we have built a framework to stop these things from happening again, what hope do we have against intelligences that are to human organizations what human organizations are to individuals?
“AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.”
Hmmm, really? What examples? Show me a machine that can do what a gnat does and we’ll talk.
The author simply invents futures to refute various arguments (all presented out-of-conext, so who knows if there's any real relevancy from that direction either).
What we might do about a super-inteligent AI would strongly depend on what that is, exactly, which, of course, no one knows or really has any idea.
So it's OK to casually BS about this stuff, but in terms of substantive discussion, much less action? No.
Why wouldn't we go through phases of increasing intelligence (along multiple dimensions) while working towards an intelligent AI? Yes, someday we may produce an AI that is leaps and bounds more intelligent than humans, but before that we will have experience dealing with various human-level and sub-human-level computer intelligences. The idea that AIs will be self-improving is kind of ridiculous -- there are lots of intelligent people, and the idea that an intelligent AI will happen to be good at designing AIs is negligibly small, much less that they will have some kind of immediate breakthrough that enables a higher level of intelligence.
It's not obvious that the path from moderate-intelligent AI to super-intelligent AI is a matter of "just adding more memory and compute"; intelligence as we know it relies too much on associative memory to think that a simple capacity increase will elevate beyond a certain point. If the intelligence takes a different form, then it's hard to make concrete statements about in any capacity, but ultimately the path will be started on by humans, or possibly emerge from sufficiently complex systems.
Either way, we're so far out from this that it is laughable to speculate. When we can simulate the intelligence of a cockroach, then maybe we can start to think about what human-level intelligence will look like. The most likely case is that at some point someone creates a system that is arguably conscious or intelligent in a meaningful way, and then we will devolve back into discussions of consciousness and identity and memory while we try to figure out what it means to think about intelligence now that we have something to benchmark against. I wrote this comment [1], that I still enjoy, in response to a similarly alarmist narrative; posed as fiction in that case rather than masquerading as journalism as in this one.
> Building an accurate emulation of the fly brain is an interdisciplinary effort that requires data, algorithms, and insight from (but not limited to) the fields of neuroscience, computer science, and systems engineering. Researchers interested in working towards this goal are invited to join the Neurokernel project.
Curious if a deep learning application that builds neural networks from gene sequences is a possibility one day. All the information is there, but it may not be possible to learn what it builds without actually "running" it.
Both are great topics for a epic book.
Had this discussion with my relative recently and recommend him to stop watching YouTube and start reading books on Introduction to ML.
We're not even to the point of superintelligent AI, but some major companies (some which start with G, just as one example) already handle situations incorrectly because of too much dependence on automation.
The author is missing or ignoring key contributors of work in this field. He even mentions Nick Bostrom, who works with the FHI which does work in this space [0]. The author's own organization does work in this space [1]. Deepmind does work in this space [2]. "Safe artificial general intelligence" is the literal mission statement for OpenAI [3]. The author may feel more conversation is needed, but it feels disingenuous to suggest that nobody is talking about this.
As for how much conversation should be happening, my understanding is that most people on the edge of the field view the current risks of existing artificial intelligence as significantly more pressing. We have the ability right now to create dangerous weapons & massive facial recognition programs with current AI. The longterm affects of biases in ML algorithms are still not well understood. Some of this is already affecting us [4].
When it's not clear at all how we approach general intelligence, and many experts fundamentally believe that our current approaches will not suffice ever, I'd argue the current level of focus on general intelligence safety is about where it should be. Smart people are thinking about this, but there is also important issues now to think about.
[0]: https://www.fhi.ox.ac.uk/research/research-areas/#1513087763...
[1]: https://humancompatible.ai/publications
[2]: https://deepmind.com/research?filters=%7B%22tags%22:%5B%22Sa...
[3]: https://openai.com/
[4]: https://www.nytimes.com/2019/04/14/technology/china-surveill...
Well but they did overtake arithmetic.
> Historically, there are zero examples of machines killing millions of humans, so, by induction, it cannot happen in the future.
Well there are also zero examples of a superintelligent AI that doesn't kill humans, does that mean that it cannot happen either ?
> No physical quantity in the universe can be infinite, and that includes intelligence, so concerns about superintelligence are overblown.
How is intelligence physical ?
> we can always just switch it off
Yeah, except at a time when we get so comfortable with an AI that turns things on and off for us. Add an "innovation" of not being able to turn the AI off by design (e.g. internet connection on SmartTVs nowadays) and there you go.
It's suprising to me that these people call themselves scientists, perhaps the quotes were taken out of context or even paraphrased.
What happens when someone develops a system that measurably makes more profitable decisions than 70% of CEOs? That isn't totally out of reach. Capital would flow to companies run by such systems. We could end up with machine learning systems running the corporate world, based purely on better ROI.
A good start on the problem would be to train on a collection of funding proposals, such as old Kickstarter announcements. Look deeper than the hype. Go out and suck up all the information you can on the founders, using credit databases. (They're asking for money, so it's a legit credit query). Do an automated background check. Then check on how the project did, five years later. That's your training set.
Well, there's your first problem. Serious AI researchers have been publicly calling bullshit on superhuman AI since at least 1987, when my old professor Terry Winograd wrote the following in Understanding Computers and Cognition (1):
"one cannot build machines that either exhibit or model intelligence behavior"
The idea that "no serious researchers are skeptical about AGI" is a total fantasy perpetuated by people who haven o idea what they are talking about.
1. https://www.amazon.com/Understanding-Computers-Cognition-Fou...
Right, show me the physics behind that one. Even smart people say unsubstantiated dumb crap. This is one of those cases.
Yet at the same time, other things have stayed the same. We still care about mostly the same things. Food, Freedom, and Fornication. And complaining about youngsters. I don't think that is going to change.
The fears that General AI are going to kill off humanity are predicated on the idea that General AI will get smarter than humanity. I posit a different interpretation of what is more likely to happen. General AI will require creativity to be able to outsmart humanity. And with creativity comes boredom. And distraction. And opinion. And argument.
I think General AI will be too interested in entertaining itself to take over the world. We've had people try to take over the world, on occasion, and it has largely not gone well for them.
But before that happens, people will start adapting AI to their lives, to the point that the line between "human" and "machine" will be blurred. So life "as we know it" will definitely "end". It will be replaced with a new form of life, one that isn't limited by only what nature can provide. One that cares about Food, Freedom, and Fornication.
The same research that prevents little problems should hopefully prevent big problems. The research to take RL to the real world is progressing quickly. I don't see a "kill us all" AI arriving soon, but "do bad things" AI and "wreck the economy" AI are quickly approaching.
Already, conventional algorithms are trusted in society, sometimes beyond reasonable or safe levels. The ethical questions posed by ethical AI design and deployment are exactly the ones our present-day reliance on (possibly poorly-designed) algorithms pose.
Before we worry about AI harming us by an improper inference, we can, in the present moment, already worry about automated systems harming us by improper inferences (e.g. improper financial models, Tesla / aircraft driving assistance); the discussion on how to limit and selectively deploy AI is no different than that for present-day algorithms.
Otherwise replicating and leasing would rake in far more than even a magic "I get everything right on the stock market" button in the same way ruling stone age tribe gives less absolute wealth than a Manhattan Lawyer.
This is what irritates me about articles like this. They state this (and then this one then comes up with a bunch of weak arguments supposedly representing the opposition) but provide no evidence of these great strides.
What we've had great strides in (as far as I know) is the application of increasing computational power for specific applications. Which is great, but is _not_ getting us nearer that goal.
This is obviously incorrect. If an AI system of superhuman mental capabilities is installed on a system with no network connection and no physical interfaces, we can completely control the harm is might create. It would be unable to directly act, of course, requiring humans to authorize and carry out whatever plan it devises. But failing to do this should be prosecuted and treated no differently from if a person or company built a tank and then permitted it to drive over and through people. The biggest difficulty here will be that our legal frameworks have no established way to assign liability and criminal culpability to any system involving software. With no legally-enforceable industry standards (like the electrical code followed by electricians, things like that) any prosecution fails as the company can either claim ignorance of the potential harm their system might cause or simply throw individual employees under the bus, claiming they were not acting on direct orders.
It is true that we can not fully specify the goal, and that is a grave concern. It's partially due to this that AI systems shouldn't be directly connected to the ability to act in the physical world unless their attempted actions go through some sort of 'filtering' first.
First, why would someone build a system like that, and how would it actually be 'smart'. If you lock an infant in a closed room, it will grow up to be a literal retard. Intelligence requires interaction and data. Next, some idiot will hook it to the net in a heart beat when they figure out they can use it to make money on the stock market. Never underestimate human greed.
Of course, a weak AI may be able to write syntactically valid code that doesn't necessarily work. That's... not much of a breakthrough.
I gave it this test:
https://www.k5learning.com/reading-comprehension-worksheets/...
A few caveats. I had to prompt a little extra on the 2nd question and give it a 'Because' to start off. It kept wanting to fill underscores as if it was a blank spot for an answer. There's a way to prevent this if I had time. The third question is "wrong". The actual answer is 'It would have become dark and cold if Ann and Frank had stayed later.', but I'd argue that it's rather open ended question and the answer is something that might have happened. The final question is really what makes this a 3rd grade comprehension test, it's asking the reader infer something not actually in the story, about how they would feel if they were Rover, something the AI is woefully unqualified for.
Question: What did Rover fetch in the water? Answer: The Stick!
Question: Why couldn’t Rover catch the rabbit? Answer: Because the rabbit ran so fast he could not catch it.
Question: What might have happened if Ann and Frank stayed later? Answer: Rover could have learned that not all rabbits love to play with sticks.
Question: How do you think Rover felt about not catching the rabbit? Answer: He was sore and he must have had fun later because he went in the back yard and hid out in the trees.
While we worry about all kids of things at the macro level microscopic organism continue to evolve. Any given Monday the blind clock maker could deliver an invisible bug right into our laps, one for which we will have no defenses.
Our interconnected world will mean this bug will easily jump population centers.
The devastation we could see if such an event occurred could be of unprecedented scale and breath. We could lose one quarter or more of the population of this planet given the right conditions.
So, yeah, AI could, I guess, go rogue. Yet the real threat isn't anywhere near what we can see and touch.
In fact, we might NEED real AI to save humanity when and if that super-bug emerges. Think about that for a moment.
Here's what we know:
- Evolution is real (stating the obvious just in case there is a question)
- Natural selection survives organism that are able to deal with the environment within which they have to exist
- Antibiotic abuse is rampant (again, stating the obvious)
- Such abuse provides the mechanism though which evolution will be able to create the aforementioned super-bug
- Once that individual emerges our defenses will be ineffective
- The world is massively connected, which means that super-bugs can travel from population to population like never in history
- Stopping super-bugs will require such measures as stopping society as we know it from functioning (no travel, local or international) and maybe even having no choice but to quarantine and allow a population (of humans) to die in the interest of stopping transmission. If a bug has a 75% kill rate and we don't have a cure, do you allow anyone from the affected group to be in contact with anyone else?
So, yeah, is it possible? Sure. Probable? I guess. I am not sure there's a deterministic function one can use to predict the outcome of evolution, even at bacterial levels. My guess is it could take a thousand years or it could happen next Monday. I don't know.
"Many Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong"
I dunno, I might listen to the AI experts on this one.
Also the statement presumes that self preservation is the super intelligent thing to do.