A case study of Toyota unintended acceleration and software safety (2014) [pdf]
course.ece.cmu.edu
course.ece.cmu.edu
The watchdog is the last-ditch failsafe, in a safety-critical system, and they basically disabled it because it was inconvenient. In some businesses (aviation?) you could probably be convicted of something for doing that, whether or not it ever hurt anyone.
You will just lose power, as though you let off the throttle fully, during this time.
In fact your car turns off fueling when you let off the throttle fully to conserve fuel.
I think the only danger is that if you were going full tilt on a sports car, and the throttle failed close, you will get lift off oversteer probably.
This is why well engineered vehicles have a separate computer for everything and not just one little chip.
You don't have to stay infuriated anymore.
The computers for in-car-entertainment is different and decoupled from the one for engine and throttle management.
As for the bloat, my observation (which could be wrong) is that Toyota seems to have comparatively less bloat.
Somewhat unrelated to this particular case, but terrifyingly, the network isn't and CAN is one of the least secure BUS's imaginable.
Can you elaborate on this?
We choose to ignore it because it's rarely exploited, but by that logic, we should just ignore all zero-days.
Learning the lessons from the aviation industry all over again.
The CAN bus is what rules in cars, and infotainment systems are generally isolated and off of it. In some cases infotainment can read the CAN bus for things like pulling speed to display on GPS, but generally cannot place items on the bus
So, he sat at every light with the brake pushed hard to the floor, trying to keep it from plunging into the intersection. As a long-time stick shift driver, I asked him why he didn't just shift it into neutral.
What if the engine free-revs and redlines on neutral?
Maybe turning it off might be a safer alternative.
> I would take a blown engine over flying into an intersection any day of the week.
And this is why we need to build experience. Young drivers very often are confronted with decisions like this but their mind phrases it as "Blown engine that I do not have to funds to pay for in Dad's car, or just hold this pedal firmly and everything will be all right".Unfortunately, the problem with experience is that the lesson comes only after the test.
But almost any engine with electronic ignition (so early 80s or later for a standard car or motorcycle, with plenty earlier having that feature) will cut spark to control RPM as needed. Unless you just start getting ignition breakup at RPM (not enough dwell time for the coil).
For example pre 1900:
https://en.wikipedia.org/wiki/Hit-and-miss_engine
Later:
https://www.jalopyjournal.com/forum/threads/mechanical-rev-l...
1970s electronic type:
https://www.428cobrajet.org/id-governor
Even my push mower has a rev limiter, it just gets called a governor on that engine.
The majority of large American engines used the valves as the rev limiter. The valve springs were so weak that past a certain RPM the performance of the engine falls off so fast it isn't going to rev anymore, even in neutral. The valves never fully close, so performance is very poor. This is well below the mechanical limitations of the crankshaft, so nothing is really going to happen. Once interference engines showed up as common place in the market, it became mandatory to have a rev limiter. This happened as electronic ignition and fuel injection became more common, so it was logically incorporated into the ECU as time went on.
Not to drop old man wisdom here but back in the day power brakes were an option that you could pay for.
The next day my friends e90 BMW m3 power steering boiled over. And another friends F80 M4 had multiple power steering failures. They weren’t exactly happy about it either.
For throttle stuck fully open, going very fast on the highway, yeah you're gonna redline, but it's probably better to redline in neutral while you pull over than to turn the key off while you're still driving.
> Maybe turning it off might be a safer alternative.
Young drivers might not understand the difference between the powered-down and the pull-the-key-out positions. There is a real danger of locking the steering wheel in this case.Which one are you optimizing for?
Likely he just had a high idle due go some vacuum leak or whatever, as you said.
The statute of limitations has passed on all of my “she’ll be a’ight” endeavors, but they included having to manually apply throttle to keep the engine lit, driving on partial service brakes, driving without a working charging system, having to push start the car, reckoning speed by gear and RPM, and fueling based on ded reckoning with an INOP fuel gauge and odometer.
Obviously if you want to drive like a jerk you need the car to be in tip top shape so that you have the most freedom to make control inputs but if you just want to putt along and drive in a mild mannered way having a bunch of mechanical oddities that constrain what you can do isn't a big deal.
However if the throttle is stuck and you are in neutral you'll end up revving at redline until you turn off the engine. Not the end of the world momentarily but you don't want to leave the engine revving that high for a prolonged period of time.
Another boat had a diesel. This boat had serious flooding and I went over to help ($1M+ boat, owner had backed it hard into concrete pier and not noticed a transom crack that opened). Engine would not shut off (electrical all shorted out). That diesel engine was running full tilt almost totally submerged - was pretty impressive. Eventually someone got a mask and snorkel and dove to fuel cutoff before it ingested a bunch of water through air intake (which I'm sure would have stopped it). With diesels I always liked knowing where manual fuel cutoff was after that.
Nowadays diesels actually come with throttle bodies for emissions reasons which I think also will serve a safety function against runaways.
from what i understand it's mostly to have a control over the air fuel ratio.
It’s for emissions
These were massive engines. The block was bigger than my whole car. Terrifying watching one of them run away. They tended towards "violent, unscheduled disassembly" if they couldn't be shut down.
EDIT: I read that on some cars you can have problem to change to Neutral without brake, so even if brake fails I guess your only option is slowly pull the emergency brake.
The throttle return spring is rather weak, and it doesn't take much to gum it up. A little force the other way will usually unstick it.
Apparently it can happen with carbs in general. I knoe people tgat had issues switching this engine off in the desert because the fuel in the carbs evaporated and continued to be pulled in the engine. Which was hot enough to to ignite it for a while...
I kind of wonder if we're heading the wrong direction with drive by wire cars. In aerospace it makes a lot of sense, especially when you are super dedicated to safe coding. But in a consumer vehicle? Not so great. Then again, Boeing hasn't been doing great with it either.
In my mind, all three are done with more or less similar rigour conforming to strict standards. Yet, exceptions are seen in all of them.
Or to rephrase: it would appear they made no credible effort given the types of defects that occurred.
According to the PDF, they are not actually required to adhere to any software standards, and they did not always follow their own coding rules. An internal email admitted that "technology such as failsafe is not part of the Toyota’s engineering division’s DNA". They didn't even have bug trackers, config management OR COMMENTS in the 250k+ lines of code that were looked at. The software was full of bugs and terrible coding practices, plus the CPU was routinely pushed way too close to 100%. The ETCS code in question also had no unit tests, but it would be impossible to have them anyway due to their use of recursion in the code, which is also not supposed to be used in safety-critical systems.
Do you disagree that the software for medical devices is written badly, or are you just saying that it should be written well? I don't think the latter is in dispute. For the former, https://www.theguardian.com/technology/2017/aug/31/hacking-r...
I can't believe I didn't think of it, but... I didn't realize that older throttles could get stuck.
I do know that power steering is literally the difference between smaller women being able to drive and not. I had an old Ford Aspire that a couple of my friends just couldn't drive because they couldn't turn it if it wasn't moving at road speeds.
Only because once it's a standard feature other engineering departments start doing things that requires the system to be there in order to get good results.
The steering geometry that keeps modern (like mid 00s on up, the "wide tread, narrow sidewall" era) cars from wandering on crowned roads substantially increases steering force required.
It's not like small women didn't drive 60s barges just fine without power steering.
> It's not like small women didn't drive 60s barges just fine without power steering.
Those vehicles had almost no castor (you mention that) but also had huge steering wheels. The advent of power steering led to the evolution of other components actually requiring it.Do you mean side mirrors? I recall when you needed to lean over to the passenger side to adjust it.
One thing I absolutely hate these days is cars without a shifter lever. The stupid knobs that killed Anton Yelchin, the new Ford junk that won't shift if the car doesn't have battery, etc.... Also, electric parking breaks.
I had a 1991 Taurus SHO (manual) that did this. I'm pretty sure it wasn't the throttle assembly, but the cable would bind somewhere and it would keep the throttle open. I could never figure out where it was binding and neither could any mechanics.
I usually had to pull over to turn the car off and back on again.
The SHO accelerated very fast (at the time) so thankfully it never happened when I was facing a wall.
The reason we all keep seatbelts bucked in flight even when the sign is off these days is Qantas Flight 72, which took two sharp nose-dives in flight, rebounding passengers and crew off the ceiling and causing severe injuries. At the time, initial reporting was that they'd caught two freak mid-Pacific downdrafts, which had me nervous about flying for awhile.
Turns out, the root cause was bad angle-of-attack data coming in from a faulty inertial measurement unit, which should have been caught by a safety system but proved so aggressively faulty that the safety system was overruled. It resulted in the plane's avionics becoming convinced it was flying belly-on into the wind at cruising speed and about to stall, so it did the thing it's supposed to do: pitched down hard to stall-recover.
Don't ask me why I'm more comfortable with the story being "Sometimes computer engineers screw up" than "Sometimes our understanding of the physics of aviation just breaks down," but I am. ;)
... but I've also done game development, where maximum performance in minimum time is absolutely essential, and I wonder if it was actually feasible to maintain the performance the control system needed without global state.
There's also the possibility of a culture clash. I've noticed that in embedded systems architecture development, global variables aren't frowned upon nearly as hard as they are in enterprise software, mostly because of the industry history of doing more with less. So I imagine a software culture risk developing from getting a bunch of hardware guys on a project where they now have access to microcontrollers with more memory than ever, but their risk tradeoff mindset is still "take the risk to squeeze every byte because we might run out."
It's been a while, so maybe there has been a reckoning since.
This slide is written by someone that has a bias.
I have not seen any independent confirmation that the Toyota problem was caused by a software issue (and indeed the slides mention that NASA looked at it and did not find such an issue).
I truly believe that it was human factors and not a computer failure.
I've been in similar situations and had a acquaintance drive through a wall because of human factors. In our case, the parking lot was relatively flat but had a drainage channel at the front. So, you could put your car into reverse, release the brake, and your car would roll forward. Quite unnerving if you are driving a stick, but your brain processes it because you are used to rolling in the wrong direction. If you are driving an automatic, you smack the accelerator, jump across way too small gap between you and a building, and bury your car trunk in an office wall.
Nevertheless, I'm also quite happy that Toyota got smacked as their development process was TERRIBLE. They 100% deserved what they got even if it wasn't a computer bug.
(and was only a problem because they didn't handle critical variables correctly, by having mirrors of the values that could be compared to protect against various types of corruption)
The material from Barr is worth reading too (google it), and if you want some amusement:
https://www.embedded.com/why-every-embedded-software-develop...
https://www.embedded.com/a-rebuttal-to-why-every-embedded-so...
> Copyright 2014, Philip Koopman. CC Attribution 4.0 International license.
• Poor isolation of task functions
• “Kitchen Sink” “Task X” both computes throttle angle AND is responsible for many of the failsafes (same CPU, same task). Brake Override function in 2010 MY Camry is in this same task. [Bookout 2013-10-14 AM 80:5-82:16]
• OSEK RTOS not certified; 80% CPU load (> 70% RMA limit) [Bookout 2013-10-14PM 42:6-25] [NASA App. A p. 119]
• Many large functions – 200 functions exceeded 75 lines of non-comment code [NASA App. A p. 23]
• Reviews informal and only on some modules [Bookout 2013-10-11 PM 29:24-30:5; 2013-10-14 49:17-21]
• No formal specifications [Bookout 2013-10-11 PM 29:24-30:5]
• No bug tracking system [Bookout 2013-10-14 PM 49:3-50:23]
• No configuration management [Bookout 2013-10-11 PM 30:7-10]
That doesn't look good. No bug tracking system? Large functions with no comments? Certainly not best practices as I've ever seen them described.
> Large functions with no comments?
I don't think that's what the citation says. It says the functions had more than 75 lines of code not counting any comments. It doesn't say there were no comments in these functions.
Nitpick aside, I'm sure we can all agree that a large number of such lengthy functions is not good practice.
But that's a big assumption to make for people panicking.
The correct solution to a runaway engine is to slap the car into neutral and then apply brakes and pull over if necessary. But that's not something most driving schools teach nor is it something your average person will practice.
2-3 in practice.
Most brakes don't actually lockout engine operation, because there are cases where you want the engine to rev while the brake is applied.
Just because at first blush it seems the use case is pointless, doesn't mean it is.
You think the Camry was designed for track duty?
[0] http://askakorean.blogspot.com/2013/07/culturalism-gladwell-...
That's really not true at all. Generally, as braking systems pick up heat, they become continually less and less efficient at slowing the car down and will eventually stop working entirely. This will be especially true when fighting against an engine running at wide open throttle. It's called brake fade.
The problem is having the presence of mind to do that when confronted with such an unusual situation
Was the key not usable? Ie, turn off vehicle?
Was neutral not available?
To get my car to go I need key in the on position AND the gear in drive. I can go to neutral and turn off key anytime.
These stories of barreling down the highway with key locked, throttle locked, gear selector locked seem bogus.
Remember, when you turn car off you are stopping, that's your goal. With brakes you don't need to "drive" that far once car is off. Turn off car, press on brakes, done.
Why is a crash guaranteed? I've driven with a car off (manual start when alternator out). It works OK (tight turns excepted without power steering)
To the best of my knowledge, steering locks only engage when the key is removed.
But I wonder what would happen if I was travelling and I just pressed the start/stop button, if the car would actually switch off.
Neutral is the solution. But when panicking you’re unlikely to use something you likely never use outside of an automatic car wash.
Seriously, when will the excuses for dirty code end? Whatever you think the business is, clearly killing 89 people is not it. Use a linter and remember your contribution to the business as a programmer is programming properly.
If programming is below you go do something else. If you have management aspirations go interview for a management job. If you don't like coding properly go code something else or consider another career.
Write proper code and stop talking about how dirty code is good for the business. Not only it is not true, nobody wants to hear about it, and nobody should have to deal with your 4d jenga tower in a professional setting, goddamn it.
All the software stuff was discovered as a result of lawsuits. The software was atrocious and may have been capable of causing issues. But I don’t think anyone ever proved it did to any reasonable degree.
Luckily for everyone involved my grandfather knew what to do and slapped the stick into neutral before we all went careening over the rocky crossing. I was a bit slower getting the clutch in and the brakes on being a new driver. It was pretty weird feeling stopped in the middle of a stream with brakes fully on and the engine roaring. He then reached over and turned off the key, making me feel like an even bigger idiot for not thinking of it.
The point is if you're not mentally prepared for a runaway engine it's hard to figure it out on the fly in the mere moments you have before your doom.
Follow on note: Grandpa gave that truck to my uncle a few years later and my uncle managed to blow up the engine almost immediately. Turns out some of the head bolts were not fully tightened down at the factory but my grandfather drove it so gently that it wasn't a problem except for a persistent but low grade oil consumption issue for its entire life.
Of course you just put the clutch in and nothing to worry about.
If you're trying to go around corners as fast as possible in a front-wheel drive car you're going to drive with one foot on the brake and one foot on the gas and will apply the brakes and the gas at the same time.
The article also says that global variables will kill you, putting variables on the stack will kill you but doesn't say where it is safe to put variables. I can't imagine the heap is any better, that a garbage collector isn't going to pause and kill you, etc.
As an Arduino enthusiast I see statically allocated (global?) variables are frequently the way to go for an embedded system, the frontier is to be able to prove the correctness of what you're doing.
Sure, but that's not normal driving and even though it should be possible this situation should simply never occur outside of a racetrack, a regular driver will move their one foot from the gas and move it to the brake.
It is better to leave options open rather than to lock things behing explicit configuration in the abscence of readily available documentation.
Which I assure you, Service/Operators manuals are not.
Global variables in that kind of codebase might be -really- hard to keep track of how you are accessing and writing them :/.
This gets really fucking nasty when you're doing things like setting it to volatile and potentially accessing it across threads and you end up with a partial write and then doing a read, etc.
I'm sure the axiom of "it's never tested as much as we want" still probably applies here and that's slightly terrifying.
also FTA: 2272 - global variable declared with different types
“Spaghetti code”: Incomprehensible code due to unnecessary coupling, jumps, gotos, or high complexity
Yeah that sounds fun. really fun. This is why I purposefully never work on any system that could kill someone. I just can't imagine being in charge of something like that and not wanting to go insane with knowing my code wasn't checked enough. or I wasn't careful enough.
(It's specifically Matlab Simulink for those playing along at home)
The thing that I never got is why there are plenty of high level languages that have 'global' more or less as the default. This is asking for trouble. Side effects should be very carefully introduced and reasoned about and kept to an absolute minimum, not be the default sauce to sprinkle across your codebase, the possible number of states increases very rapidly if you do that.
1. Please don't do circus on public roads.
2. If you're on private roads, well, you know, front-wheel-drive cars tend to handle poorly in fast corners. Do yourself a favour and get a RWD or AWD or something.
This does not seem like a valid use case for driving on public roads.
Driving in the snow often requires this because the brake acts as a poor man’s diff lock which gives you more traction. You can see the effect at speeds as low as 30kph in snow – a nice slow speed even in the city.
You just hold the brake (lightly) with one foot, and gas with the other? This doesn't sound as useful as putting the car in 2nd, or rocking back and forth?
Neither of those cause the wheel with more traction to get any torque. Applying the brakes lightly will. Rocking and manually preventing wheelspin with the brake can be combined.
In theory, traction control will as well, but traction control may also intervene and decrease power right when you need it as you're getting unstuck, so there's potential merit in turning it off and preventing excess wheelspin with the brake yourself.
Source: I learned to drive in Alaska, and I have used this technique.
Thanks!
If people don't come out of a system interaction smarter than they went in, you're not doing it right. You're writing checks their continued ignorance will increasingly extort hogher costs from, and which will culminate in the formation of a lever of control in the form of manufacturers making decisions for people, usurping ultimate end agency.
I just don't buy that that's how the world should work. You can't build a better world without building better, more knowledgable people. We engineers spend too long making machines that do things for people without making clear why they exist and how they work.
I'm done with the "for their own good" BS.
We should empower people, not manipulate them through engineering usecases to/to not accommodate.
https://www.mazdamotorsports.com/2019/02/07/a-better-faster-...
And Spec Miata is a pretty terrible example anyways... at the point where you're dumping 10k into a $500 motor I don't think you can talk about stock anything...
Worth noting that 'statically allocated' and 'global' aren't the same thing. Whether they're making that distinction in the powerpoint I'm not really sure because it's more of a language specific thing and the document isn't quite that technical.
As for how you do that, in C/C++ you can add `static` to a global variable declaration to make it "local" to the file/compilation-unit it is declared in. The variable itself still works the same from within the file, but a static variable cannot be "seen" outside the file even with an `extern`, so it can only be (easily) modified from functions inside the file. Doing it this way allows you the big advantages of static allocation (in many cases the only real option) while still allowing you to encapsulate the variables within a sane API that the rest of the code uses.
Edit: Actually page 39 references this, saying that 89% of the variables are not locally scoped. There are still ways to achieve some level of encapsulation even without using `static` but it's effectiveness drops a lot since anyone can just write an `extern` line and ruin your day...
If you want a race car, buy a race car.
Why do you need the brakes? The nearest freeway on ramp to my home is unusually short, and it's a 270 degree turn. I've had to get very good at getting my vehicle to accelerate as quickly as possible around that curve so that I can merge safely when the on ramp ends.
I don't need the brakes to do that, in fact the front brakes in most vehicles are usually much stronger and the braking bias is usually weighted greatly in favor of the front brakes. So again, I really don't know why you need the brake, especially in a front wheel drive car... you're just fighting the engine and the brakes can usually win that fight (except for in high powered rear wheel drive cars where the rear brakes are weak and the engine can overpower them).
This isn't relevant for your example as in your example, you are entering the freeway on-ramp at a low speed and accelerating throughout the entire corner. Navigating a corner at highest possible speed means you're arriving at the corner at a high speed and must decelerate in order to make the corner.
It's not immediately clear to me why you would use gas and brake at the same time in an automatic transmission vehicle. The only reason I can think of would be to keep the revs of the engine in the powerband and prevent the car from downshifting so that you can immediately have full power when you accelerate out of the apex. In a manual transmission vehicle, there's a technique called heel-and-toe where you blip the gas pedal with your heel to rev-match the engine on your downshift while braking with your toe and depressing the clutch with your left foot, but you're never introducing force to the front wheels that would counter the force of the brakes.
When cornering hard, because of weight transfer, the inner wheel loses almost all traction, which makes it impossible to send any torque to it without wheel spin. This means that you can't really accelerate out of a sharp turn with an open-diff vehicle, because an open diff always sends the same torque to both wheels, i.e. if the inner wheel spins without much resistance, that resistance equals the torque sent to the outer wheel.
A poor-man's-LSD-solution to this is applying a bit of brakes while also giving it a lot of gas. The brakes prevent the inner wheel from spinning freely and add artificial resistance, which increases the torque available at the outer wheel.
Ideally you'd only want to apply the brake on the inner wheel, which is what some cars actually do automatically as a fake-LSD solution, but applying both and wasting engine power is better than nothing.
As a professional in the embedded and safety critical systems space: No, they aren't "the way to go". You often end up with a small number (think "singletons" for OO people) but the rest can be done properly with local variables or more limited scope variables instead of garbage global variables to manage complex system state.
Function-scope static or file-scope static variables are the best place for variables in an embedded system written in C. They don't pollute the shared namespace, nor do they require allocation and de-allocation.
In an Arduino program, for example, you might have something like this, with a sketch.ino file:
#include <stdint.h>
#include "MyFile.h"
uint32_t Bad_Global_Variable;
// This doesn't conflict with same name in MyFile.cpp
static uint8_t state = 1;
void setup() {
setup_myFile();
state = 0;
}
void loop() {
loop_myFile();
}
a MyFile.h header: #ifndef _MY_FILE_H_
#define _MY_FILE_H_
#include <stdint.h>
void setup_myFile(void);
void loop_myFile(void);
#endif // End include guard
and a MyFile.cpp library: #include "MyFile.h"
static uint32_t MyFileScopeVariable = 0;
// This doesn't conflict with same name in sketch.ino
static uint8_t state = 0;
void setup_myFile(void)
{
MyFileScopeVariable = 1;
// Can't access or have conflicts with myFunctionScopeVariable here
state = 1;
}
void loop_myFile(void)
{
static uint32_t myFunctionScopeVariable = 0;
MyFileScopeVariable++;
myFunctionScopeVariable++;
}
Statically allocated variables are definitely the only the way to go.Statically allocated global variables are easy and available, suitable for small embedded systems written by one person or a small group of collaborators who can be expected to know about every variable in the program or at least to be able to refactor their code if it collides with an existing name. Adding a prefix to your statically allocated global variables (eg. 'mf_state' for a global variable in the MyFile library above) is another way to reduce collisions, but isn't compiler-enforced.
Statically allocated file-scope or function-scope variables are a best practice in an automotive ECU scale projects, and can reduce issues if you've got lots of vendors each contributing code and not a lot of visibility between projects.
Mario, when you do this move, is your third foot on the clutch?
Heel-and-toe shifting is used before entry into a turn while a vehicle is under braking, preparing the transmission to be in the optimal range of rpm to accelerate out of the turn.
There are vastly more people who are just bad drivers who use both feet like this out of poor skills or bad habit. Afaik this was the main cause of "unintended acceleration" with panicked drivers.
No you won't.
> As an Arduino enthusiast I see statically allocated (global?) variables are frequently the way to go for an embedded system, the frontier is to be able to prove the correctness of what you're doing.
When you have 2kB of RAM to play with you haven't got a lot of stack or heap to sling function parameters around on, so it's Through The Looking Glass and you do it all backwards - everything goes in a global unless you have a really really good reason for having local scope (loop counters would be a good example).
If you’re going around corners as fast as possible in a front wheel drive car, sell it and buy an AWD or RWD car that doesn’t understeer.