Geoff Hinton Wants Computers to Think More Like Brains
wired.com
wired.com
Unless there is a condition in the environment that causes a systemic failure involving large number of self-driving cars at the same time. This happens all the time with other types of software. People shouldn't assume that machine-learned models are magically different and ignore all common-sense rules we apply to safety-critical applications. In your daily job, would you be willing to bet your entire car on a black box piece of code that seems reliable, because it worked most of the time in the past?
Also, recall that flash crash of the stock market. Imagine something like that, but with lots of cars on a highway. If you don't understand "how a self-driving car does it", you are unlikely to install appropriate safeguards.
PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
It can be simultaneously true that we cannot describe in a causally deterministic way how a system works while also determining with some confidence that the system is performing as desired. This is true of all human systems.
So if you don't know how people drive, but have enough confidence to let them drive based on performance in evaluation scenarios, then similarly we should be able to let machines drive assuming they perform equally as well as humans over the same contexts without knowing how they drive either.
2) you’re describing how a system can crash. Driverless cars don’t share state, besides, I guess, vast scale astronomical and weather conditions. It’s not likely something would cause all cars to crash because of a collective hiccup. There may be a set of circumstances that consistently cause individual cars to crash, but that’s not really the same thing. And we can understand how these events are processed, so unlike people, we can adapt for future repeats of this scenario.
Errors or malicious attacks upstream could lead to disruptive or even dangerous behavior by a lot of cars at the same time.
This happens to human drivers every year. Snow, freezing rain, fog; these adverse weather conditions can lead to large numbers of collisions at the same time. Expecting self driving cars to never have a problem during rare environmental corner cases is to expect them to be vastly superior to human drivers all the time. If you're waiting for that day before you consent to self-driving cars sharing the road with you, you'll be waiting a long time.
Well, self-driving cars can currently handle none of the aforementioned conditions. That's why they are being tested where those conditions don't exist, and that's why we're about 20 years from widespread adoption. It rains and snows in much of the rest of the world.
Yes, I live in the rest of the world and my family has been affected by fatal collisions brought on by adverse weather. I remain skeptical that self driving cars will be able to handle all of these things. I do think they'll be okay for mundane driving where the biggest risk is driver distractions, however.
I would like to see cars and maybe even roads made smarter in order to make driving safer. If all the cars on the road were part of a wireless network and they could signal to one another when they're braking or turning, then other vehicles could respond more quickly.
Heck, these systems could even be included in phones and pedestrian crossings so that cars could be signalled instantly when a pedestrian steps into the road in order to cross.
BUT (and there's always a "but"), the problem is exponentially hard, and it can't be solved with today's technology.
I agree with you that widespread adoption will require modification of roadways, and potentially also segregation of autonomous and human-controlled traffic streams.
Cars sliding out of control on a snowy road is not a systemic failure.
If you have an entire fleet of self-driving cars using the same (or similar) technology, the exact same condition could cause all of them to misbehave in the exact the same way. This could happen simultaneously with a large number of cars. Or could be something that propagates in a cascading manner. (A car breaks, turns and spins out of control. The next car detects that, breaks, turns and spins out of control, etc.)
If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?
People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving prioritizing safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it was designed and refined to be used by humans.
It's astounding how many of AI enthusiasts fail to see these issues and differences. Traffic system isn't a board game.
A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them.
> People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans.
Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations.
> Traffic system isn't a board game.
I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject.
When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives.
We adapt.
Some cars today already have IR sensors (night vision systems) installed in them, and include pedestrian detection monitors as well. Cars don't have to be full on self-driving to have a large increase in safety and accident prevention.
On the other hand, noone was ever run over by a mistyped URL.
Human intervention would be required to enable the vehicles to move again.
After some real-world experience, I would want to give the emergency stop system the ability to release the hold on one layer of vehicles at a time, allowing the smart AI to drive again but at a speed limited by the "dumb" system to nothing faster than human walking speed, enabling them to clear the area. If they cleared without further incident, the next layer would be released until the whole stoppage either cleared itself or was restopped because something was still wrong.
These kinds of crashes generally have the some root cause: everybody is traveling too fast for conditions, and when one thing goes wrong, the end result is that you get a chain reaction of everyone accreting into a single crash. If self-driving cars are capable of making the same misjudgements, then these kinds of crashes absolutely can happen.
For the average non-technical person, this is pretty much their relationship with all technology.
I don't think it's exclusive to tech either. I know lots of people that have absolutely no idea how the mechanical side of a car works, but they trust their lives to one at 100km/hr because it has worked generally pretty well in the past.
Car safety has made great progress in the past 10 or so years. People don't have to know how they work under the hood as long as designers and engineers already built safety into them and these designs have been heavily tested by third parties (e.g. Euro NCAP, IIHS, etc)
Self driving cars will have to prove their safety, especially in extreme weather and non paved roads.
But another way to look at this is the fact that "cloning" the same AI brain across multiple cars is equivalent to a single AI brain controlling many cars. And that does have human analogies where you have a single person who has a dramatic amount of power. Any flaws in that human's reasoning can be disastrous.
Trump having access to the nuclear launch codes comes to mind.
In practice, the way we mitigate this is by surrounding that human with advisors and other brains to try to help shore up deficiencies. We could do something similar with AI. Have a couple of differently-trained models and compare their outputs. If they disagree wildly, then try to gracefully shut down the system.
Google something like "top causes of automobile accidents". No list I found had exogenous causes (rain, snow, etc) in the top 5. For machine driving to top that isn't going to be that hard given that we are our own worst enemy on the road.
There just aren't reliable statistics on what the safety of this tech will be at this point in time. All we have is extrapolations with lots of bias and little independent review. Maybe it will eventually be worth it, but that experiment will come with a great many human deaths and we can't simply brush aside the ethics of that. I personally think it won't prove to be safer than humans for many decades. This is far more of a complex problem than a lot of the developers think, and I believe there's a case of engineering arrogance going on here as well.
On the surface it seems like we have something very close to a viable product, but the remaining little details and issues are important and represent an enormous amount of work, where we are going to see diminishing returns and enormous difficulty making a system that is flexible and robust enough to handle all the required situations more reliably than the average driver on its own for thousands of miles, scaled up to millions of units around the world. They will be good enough for niche applications, but the dream of a general self-driving infrastructure is pretty far off.
You don’t need to be better than humans for all situations. You need to be safe. Humans are generally well above that threshold, except for when they’re drunk, high, tired, distracted, or otherwise stupid. It doesn’t really make sense to compare deaths per mile for cars vs attentive humans in stable conditions. Both can honestly be reduced to the point at which they’re practically 0 unless it’s the victim’s fault. We do need to be attentive to edge cases that cause cars not to be safe, but we can also observe the overall pattern and draw conclusions on how likely unknown edge cases are to be encountered.
Local ones seem unlikely to cause casualties larger than events that fool large numbers of human drivers, like sinkholes or malfunctioning traffic lights or black ice on a freeway. Some of these will probably happen, and the systems will be improved, and they'll happen less and less often.
Global ones could be very large, and there are several instances of global, cascading failures in other systems like telephone or data networks. There are known ways to prevent many classes of failures, but getting to 100.000000% is hard.
Any system that requires a human operator to passively observe a boring process for hours at a time, yet snap to attention immediately and without warning to take control and prevent human deaths is simply insane. That's the diametric opposite of how a safe system should be designed. If the human is required for safe operation, they should just be driving at all times to maintain vigilance. I don't see people talk about that much. The assumption seems to be that AI will quickly get better and the human back-up will not be needed--that the human is kind of a temporary workaround. Well I've seen plenty of engineering projects where the temporary fix becomes a long-term pain in the ass because the difficulty of the real solution was massively underestimated and under-budgeted. We might get stuck with these terribly designed AI-human hybrid systems for many decades as AI keeps getting pushed into the horizon and we slowly realize that this is one of those hard problems that doesn't progress the same as most other tech does.
There are pure driverless systems out there, but they don't cope well with many of the incongruities and fringe situations which are an unfortunate reality on the open road in most of our country. I think they will be confined to niche uses for a long, long time.
I don't see why this would be any different for deciding whether computers should be allowed to drive - we can just make sure self-driving cars pass a road test. We could even do better than driver's license tests and have a human driver at the wheel during the test just in case.
We let people drive because we know that they are fully capable agents able to make decisions even in case of failure. In other words, they are responsible, reliable, fully cognisant individuals, far beyond the technical ability of operating a machine.
A self-driving car is like a parrot trained to speak. Not only is it not an agent aware of its larger surrounding and potential implications of getting into unknown situations. It also has no real understanding of what driving is at all, it's even stupider than a pet. And because its software is usually deployed to thousands if not millions of devices at the same time, there is little to no variance to protect from extreme failure.
I wish I could share your optimism in humanity, but what you just claimed sounds far from being true universally.
Take your typical driver and throw in a failure situation they have never trained before (because, who has time to train them for every possible, or evens few likely failures), and it is not likely to end well.
Humans have a base line of common sense reasoning and the ability to create coherent models of the world. A human driver isn't (rare exceptions aside) going to drive up an impossible street or try to dodge a street sign mistaken for a kangoroo in West Germany.
There's nice examples of how an ML algorithm can be tricked into all sorts of pathological behaviour by choosing adversarial images, there's no baseline of reason to fall back on.
Imagine a model that has one of these fatal edge cases baked in and a thousand cars suddenly mistake a flash of light for a u-turn. Sure, one or two human drivers out of a ten thousand might drive that irrationally on occasion, but compared to a machine we can usually cope with this.
And I don't even want to imagine what happens if this turns malicious. Imagine a bad actor on a bridge above a highway with some artifact that messes automated cars recognition up and the developers forgot to put some hard fail-safe in for that piece of road.
It's worth noting the downsides to this agency too, though: Agency is what makes the stereotypical male 20yo driver speed and drive dangerously to show off to his friends, which leads to a very large number of accidents.
It's unclear what accidents this agency avoids though. I can imagine a few situations where it helps (avoiding a terrorist attack by speeding away from the scene?). But in most cases the default operation for a self-driving car will be to stop as quickly and safely as possible in unusual circumstances, and it's unclear how often this is the wrong decision.
I won't insult my intelligence; I'll ask why you trust people even though you don't know how they work and you know that at a high rate they intentionally impair themselves before driving?
Recall that time when a human trader lost a billion dollars?
https://traderhq.com/traders-cost-company-billions/
Remember when the economy tanked because everyone in finance used Black-Scholes model that seemed reliable, because it worked most of the time in the past?
This happens all the time already: there is a condition in the environment (eg when it snows in Seattle) there is a systematic failure involving large numbers of people driven cars at the same time.
People just don’t have experience (erm, “training”) with the weird unusual environmental condition and fail to deal with it. Not surprising that computers would have similar problems.
> PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
But we actually do know how well people drive. We have data on failure rates for sure. We don’t know how they drive, that is completely opaque.
Manager: Why did you delete the production database in the middle of a work-day, no less?
DBA: My decision was based on all sorts of things you can quantify, and then all sorts of gut feeling. I have no idea how I do that. Do not ask me to explain my decision, or you are forcing me to make up a story.
Manager: But, you crashed our client's entire business for a whole day!
DBA: I can't explain how I work.
(Somehow, I don't see that flying.)
I guess pointing that out while explaining yourself is a bad idea though.
I don't. In my opinion, we are all getting carried away with ourselves.
Google or not, I challenge anyone who makes a statement including the phrase "...more like brains", to define that statement accurately. In response, you will likely hear all sorts of things that sound like we know what we are doing and are very clever, but.
If we can't even ask the questions properly, I doubt we will get deliberately valuable answers. Sure, we may luck out, with AI researchers stumbling across random combinations (layered AI) that do produce useful results hap-hazardly, but I would never rely on it.
Eg. I rely on software running in an aircraft, because:
a) it works, by design, not trial and error. b) it is verified. c) we know how it works.
unless you have (c) you cannot achieve (b), with any degree of certainty. Experimenting until confident is not good enough for me for anything important.
Brains fail at times. Computers should not make mistakes.
Even if you knew how a brain works, and even if its possible to replicate it, please don't make computers like this. Software fails enough on its own without any help, even when we supposedly understand it.
With this AI stuff, even if we knew where we are going, we don't know how to get there. And even if we did get there (wherever that is), in the long run, AI is not going to be used for the good.
As always with humanity, we will find ways to use AI to extract value out of other people. This is what drives business.
No thanks, but it is interesting for fluff. A bit too voodoo for me.
I don't think this is a reasonable position to take. We are going to have to accept that if we want computers to do the things brains can do, they will also make mistakes like brains do sometimes. It's not reasonable to insist that the error rate is exactly zero. Besides, if you consider the computer as a complete system including the hardware and human-computer interface, it's clear that no complete system can ever be error-free, even if the CPU executes its instructions faultlessly.
As long as it's as good or better than a human, it's good enough to be useful. Even with "fallible" AI techniques, we should be able to make systems that are much more reliable than humans.
That's my point though. I don't want computers to do what my brain does, mainly because I have no idea what my brain does, nor does anyone. Sure, some people think we do, but we don't.
It also seems silly to try to engineer something that we know may fail (by design), on the probability that it may haphazardly produce better results (even if this is most of the time). That may be useful for things like search results, where the outcome is not critically important.
Please excuse the example, but I would be happier passing away in an aircraft accident knowing it was because although the system was designed not to fail, we did our best in terms of understanding the systems in place. That would be my definition of "sh&t happens".
If I passed away because the aircraft "AI" took an untested path of execution (there are too many to verify, unlike most regular "dumb" software), I would be less happy.
Maybe there is a balance, Eg verifiable software for the important stuff, AI fluffery for the less so. But in that case, we should not get carried away (eg. self driving cars). Remember, planes could fly themselves completely for a long time, and we still have 2 pilots for a reason.
Computers are a human's tool, not replacement.
OK, that's a valid position to take, and I'm sure some people agree, but I and many other people do want computers to do what our brains do. And it is pretty likely to happen.
> I would be happier passing away in an aircraft accident knowing it was [not AI]
If I was in an aircraft accident caused by human pilot error while pilot AIs with 1/100 the error rate of humans were blocked from deployment because they don't meet your "zero errors" requirement, I would not be happy.
I hear you, but my point is "what do our brains do?"
In the case of driving, if the answer is "drive a car", then that is a bit like "brexit means brexit" :) The definition of the thing is the outcome of the thing.
Instead, if you try to can decompose the question into a more useful form, one that can drive a specification of software, then progress can be made. However, all that has done, in that case, is a description of the outcome, not the process. (And incomplete, due to the complexity).
It is at this point where my problem with all this AI stuff starts :)
The lack of understanding of the process, rather than the outcome is why I call it voodoo, and anyone talking different is trying to impress, in my opinion.
While we don't understand how our brain works to the bits and bytes, our knowledge and understanding of how our brain functions is Growing. A fuller understanding of how our neurotransmitters work on a chemical/biological level, and an effective way to model it, will likely result in interesting advances in the field of AI
I chat with my neuroscience friend a lot, and the overlap I see between how he describes how the brain works, and general computer science concepts is surprising.
In my view it would be unethical to stop the deployment of such AIs on the grounds that we don't "understand" them well enough or can't predict their behavior in every hypothetical circumstance, because we can still test them well enough to know with statistical certainty that they will save many, many lives.
Estimates about availability have consistently been overly optimistic. This may be a case where the first 99% of the work takes 99% of the effort, and the last 1% takes the other 99% of the effort.
These are the kind of problems which have profound amounts of complexity lurking in every unexpected pitfall. We are going to discover that it's a lot more of a tangle than we thought before it becomes simple again. I think most of us here know a thing or two about that sinking feeling when you're attacking a problem and discover that what seemed like a minor pothole is actually a wormhole leading into an entire universe full of waist-high bullshit with submerged landmines, but this problem will be in another class entirely. Figuring out the brain is a species-level problem and it's going to be VERY long and arduous.
I also resent the article's assertion that "if an AI is a safer driver than a human, who cares if it's a black box?" Fallacious argument number one around self-driving cars. Hear it all the time, almost always from laymen. It's a nice way to sneak in the assumption that AI will be better than average human drivers any time soon. The way that metric has been measured has been very deceptive so far, and no AI has come close to matching human safety and robustness. Not in real life applications in the field at scale. Not even close. There's been shenanigans like discounting all driver interrupts, which by nature disguises any flaws of the AI and offloads that responsibility on a person as a sort of PR heat-sink.
We don't give ourselves enough credit for how well we handle driving. Reality is chock full of exceptions, quirks, and legacy systems that aren't going to go away for many decades, and compared to the progress we've made on AI so far it's going to be many, many times more work for us in the future to deal with those niggling details. The devil's in there, he really is.
He did, he specified the synapses are like weights. I'll extend that a bit and say the collection of ion channels on the neuron's membrane, which regulate the surface potential, will act as the classifier. They form a non-linear classifier (voltage gradiant dissipates with r^2) but still a classifier.
Statement 1:I’ve always been worried about potential misuses in lethal autonomous weapons. I think there should be something like a Geneva Convention banning them, like there is for chemical weapons.
Statement 2: You should regulate them (AI) based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person.
The intention of utilizing AI in weapons systems is to reduce human error in their application. So is it unethical to implement an "AI" in a weapons system that kills fewer people than a person would?
Seperately, I am curious why Hinton is not talking more about state representation and prediction, as almost uniformly across DL/RL researchers this is agreed upon as the next step forward. Hinton certainly is working with Richard Sutton on his RL programs at U Alberta. Sutton believes that the next step in AI systems is state representation and prediction and gave a quick talk on it recently [1].
Reducing human error is one possible benefit of AI weapon systems but there are numerous drawbacks. AI weapons could be programmed to violate laws and commit atrocities on an incomprehensible scale. They could conceivably cause the extinction of human beings or even all life on the planet. They don't even need to be sentient, let alone super intelligent, to accomplish this. If they can be mass produced and armoured well enough to resist most small arms fire, they could easily overwhelm any human armies.
The scariest part of the Terminator films is not Skynet, it's the terminators themselves and they're far more plausible than a godlike super-AI.
"ou should regulate them based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person. With self-driving cars, I think people kind of accept that now."
If the logic can be applied to one system it should be able to be applied to all systems.
I wouldn't say this is necessary the sole intention. AI will be a force multiplier as well.
Has Professor Hinton ever made a known appearance on HN?
We don't generally expect someone to freak out and start treating a car in sunlight as a house because they've seen a lot of houses in sunlight lately.
The difference between a human and ML is that a human has a clue, demanding perfect introspection is missing the point.
This borders on dishonesty if you ask me, he should know better.
Hinton doesn't mention microtubules at all. Do any of the neural net or other connected neuron models incorporate the quantum vibration effects inside the neuron's microtubules?
Edit: Nevermind on microtubule resources. Plenty on the physiology references for those.
[1]: https://onlinelibrary.wiley.com/doi/pdf/10.1111/jnc.12621
Arguably if there is a cytoskeleton mechanism for signalling or storage it's captured in some ways in changing the weights and evaluating the net, but not if it had any unique function beyond the abstraction which was already in place from the conventional view.
Studying other structures than the synapses and membranes is prudent, but Hameroff must be somewhere on the line between crackpot and intellectual poseur. Use your time on some real science instead.
Here's one option: http://www.gatsby.ucl.ac.uk/~lmate/biblio/dayanabbott.pdf
It's conceivable that quantum effects could matter in some way on very small scales, and that doesn't really have anything to do with quantum mysticism.
Oh good; although, that predicament about publishing AI papers seems pretty bad.