Autopilot: an open source driving agent
github.com
github.com
Speed control uses data from a radar and from the vision system. The path info is used to decide which targets the radar should consider important. The big thing seems to be identifying the "car ahead", if any, and maintaining an appropriate distance.
There's a hard-coded 6 minute timeout to make the driver take control once in a while.
On a good day on a well-marked freeway, this might work. It would be interesting to modify this to process pre-recorded dashcam data and see how well it tracks the road.
There doesn't seem to be a large amount of embedded data indicative of a trained model or neural network.
So, it seems that the vision algorithms probably mirror those in the common literature and represent a reimplementation or iterative improvement on existing lane-finding systems.
More reversing would probably yield greater insight :)
Which seems to indicate that they are clinically insane, given they are using a system that can easily have excessive scheduling latencies under load to steer a car.
Now that this is open source though, I wonder what their next project will be?
In the future bits of a self-driving car system may get open-sourced the way Facebook Open Compute has emerged in data centers, but first the proprietary implementations will need to pave the way in the legal world.
It's not all bad news, though: I'd love to see a free software project that converted my lawn mower into a self-driving lawn mower...
That's not clear to me at all. It seems clear to me that the person operating the vehicle has ultimate responsibility for how it is driven, whether they're personally driving it or not.
In addition, it may soon be questionable to say that an occupant of a vehicle is "operating" said vehicle if it drives entirely autonomously.
Seems reasonable at first glance, but responsibility makes only sense if you also have the knowledge and power needed for making the right descisions.
Unless the driver is expert in the self-driving car field and core contributor to the car software, he'd end up being liable for events he cannot anticipate and has no control over. In such a situation, the only actually responsible descision would be not to use the car at all.
GPSd is one example. Used pretty much everywhere, including life-and-death real-world situations. Consumer products (like your car's GPS) including it tend to pop up a "don't be an idiot, but we're not liable even if this malfunctions" screen. (The Pentagon and oceanic research operations, well, I'm the wrong person to ask about how they handle software liability, but kinda doubt we'll see class actions from either.) But the important part is that that screen is not there to protect the authors of GPSd. It is there to protect the VAR.
But like RedHat and Ubuntu, building a distribution of well proven and flexible open source software is likely the best route to success.
There are reasons why we have realtime operating systems, deterministic bus systems and dedicated CPUs for these kinds of applications.
That might be an interesting research, prototyping or simulation platform. But nothing you want to have on a real public road.
As in research projects, which this project expressly states that it is? What are those reasons? It seems like being able to test theories as quickly and cheaply as possible is the top priority. If Python, a general purpose OS, and a CAN-USB adapter is sufficient to conduct the research and satisfies those attributes, it seems like the perfectly logical way forward.
I understand why you would want to move to such systems you describe once your research has concluded and you are now building something for production, but that is not the intent of this project, at least at this stage. It specifically says so.
This is from comma.ai, who famously tested on public roads with a reporter in the car and very little prior testing ("the first time it worked was this morning").[1]
[1] http://www.bloomberg.com/features/2015-george-hotz-self-driv...
Unless you're not planning on ever releasing then a platform like this makes sense for strictly research, but Comma AI did plan on releasing their product until they got shut down.
> Wouldn't you want to use the research phase to help eliminate such bugs?
I wouldn't think so. Spending your time fixing bugs in something you realize could have been done better another way, which ends up getting thrown out, seems like a waste of time. The words research and development are often paired together because it is a two step process, where development comes after you have learned what can be done.
It's far easier to get the code working in something like Python, and then rewrite it in safety-critical C later. And the kinds of people who can write self-driving cars in Python are using very different skills than the kinds who can write safety-critical C, so it might not even be the same person.
Even so, sorry to say, the code is miles away from any decent software standard. It is almost exactly what you might call CRAP (classes really a procedure) and mixes responsibilities all over the shop. Little attempt is made to abstract behaviour, and there are differing units, and random multipliers all over the place. Using c++ and python, both of which are OO many classes of error could be avoided if any actual OO features were used, even if MISRA was not a target, sadly the code doesn't do that.
This has all the hallmarks of software that doesnt know if it's doing m/s or mph.
I think the rub is that this code appears to be intended for "production" (or at least use on public roads), given the fact that it's published by a company that has tested its products on open roads?
Why use Python when there are existing tools that are made for this type of application?
I agree though that once algorithms are developed sufficiently they should be ported to RTOS platform and this box shouldn't be permitted on open roads.
I'm extremely familiar with OBD|CAN and car networks - the embedded code alone is an essay in how trivial something can appear (sending commands to a ECU) without considering the million edge cases that make this a safe product to use.
It's not
http://newatlas.com/geohot-comma-ai-openpilot-open-source/46...
But in a nutshell:
The company was approached by agencies that had severe concerns over safety and if this would have the required regulations in place.
So the company folded and released this instead.
And then one day a one-in-ten-million event that never would've surfaced during testing does happen and the OS crashes or the interpreter hangs or some non-determinism causes a period of non-responsiveness or a bit gets mangled because you don't have any redundancies and solar rays are thing or ... and someone dies. And of course that didn't happen in testing, because all of those things are rare possibilities.
And then it happens again.
And again.
And then you're in court being sued for millions. And a bunch of industry experts who do build their systems with redundancies and do design their systems with a safety-first mindset come to the stand and rip you apart for not taking even the most basic precautions. And when it comes out just how many best practices you completely ignored, you're mostly just hoping beyond hope that all of the legal problems stay on the civil side of the civil/criminal divide.
Or, to say it in a sentence, "because the sort of exceptional circumstances that safety-critical software needs to handle with grace are very difficult or impossible to account for with a desktop machine running interpreted code on top of Linux."
But I also feel like you're being a bit hasty. Obviously a Python script isn't going to turn into a Tesla overnight. But maybe it'll help you find a few bugs before you throw all that time and effort into building the real deal. When I look at a Github repo that claims to be a self-driving car I don't say to myself "yep, looks production ready," I say "Cool prototype, now let's break it." To me it seems entirely reasonable to get a working Python prototype 90% of the way there and then send it off to the OS programmers to design something that actually meets the concept of "production ready".
I think w/ self-driving cars there is an interesting ethical question. The full auto cars on the roads today definitely aren't production ready, but they also have constant safety drivers. Probably even ACC systems are tested in the wild with a safety driver before production.
Basically, "is the safety driver sufficient to justify running prototype software in the real world?"
Intuitively, though, I think that buying/using the software is tantamount to accepting its imperfections, so long as they are adequately (factually) presented to you beforehand. You're signing off your ability to make your own decisions, but are still responsible for them.
Also note that interpreters have their place in safety-critical aerospace as well: some satellites run Forth.
I disagree with the 'more' - how many lives are at risk with a SpaceX failure vs. a self-driving car failure? This is even without multiplying by number of users.
The existence of a component that is difficult to analyze for safety doesn't justify ignoring well-established safety engineering techniques throughout the rest of the system. That attitude would have us throw out seat belts and snow tires just because the ACC system might be buggy sometimes.
The reason the "realtime" designation exists at all is that, in a realtime system, it is possible to say what the worst-case timing is for any operation, and to guarantee by design what failure conditions can and cannot happen.
You cannot guarantee in a python program the timing of your GC. You can't even guarantee that the whole thing won't fall over with an exception.
And we've already had this problem with the Toyota "unintended acceleration" bug, in the (far simpler) electronic throttle system: https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_... - and that was using C, but not in an appropriate style (MISRA or similar).
(89 dead, cost Toyota billions)
[edit] I'll add this conversation stuck out in my mind simply because I was a bit surprised that no one knew what I was talking about, that's all.
The question is naive and has a dangerously ignorant view of what "engineering" means. But it has been properly and usefully answered by others.
(sorry for a meta comment rather than a substantive response, but I feel strongly about this).
The reason that comma.ai was able to get a working prototype quickly (unlike much bigger and well-funded companies) is that they use an end-to-end deep learning model. The model directly learns to issue commands to the car based on raw image data. The problem with this approach is that neural network models are a blackbox. They can behave in completely unpredictable ways.
Suppose you make an autopilot in JS running on Electron on WinXP, running on VM on top of Puppy Linux Live CD. And you still manage to prove that your system is 100x more reliable than human driver. Should it be dismissed just because we don't agree with technology stack? The latency of this monstrosity would still be lower than human driver and maybe they would stay lower throughout the operation.
As an example, I worked on an embedded system that controlled electrical motors that was hard real time. The fast task time interval was once per millisecond. No matter what, that got called by the RTOS exactly 1000 times per second. When it didn't finish it's job in time, the result could easily wreck real world items or cause harm to people. Nobody even considered using interpreted languages in that project. The fast tasks all had provably run in much less than a millisecond. That means no loops that could be unbounded, no memory allocations, no recursion, no writing to flash, anything that was even slightly unpredictable was out.
So, even if you could prove that your system caused less accidents than a human driver when it was running well, it would be impossible to do an analysis that defined under what circumstances the system would be running well. Given that, it would not be allowed in a well-engineered real time or safety critical system.
That's the rub: how do you prove that? If your software stack is 30 million lines of code that was written by god knows who, I would argue it's nigh impossible without releasing it and seeing what happens, which seems morally irresponsible and legally negligent. If you follow strict rules in coding conventions and algorithms, it's easier to statically verify code is probably correct.
We entrust humans with operating cars right now. Humans in a sense are running a general purpose OS.
But you couldn't be more wrong. A trusted system is one which you have to trust. No representations are made about whether you should or not.
Humans are frequently trusted systems.
https://itunes.apple.com/us/app/dash-train-self-driving-cars...
https://play.google.com/store/apps/details?id=ai.comma.chffr
Sorry, maybe that was an inappropriate aside but I know I'm loud & proud team GEOHOT and have yet to see the Giants in the industry really show they can fend off a determined Goliath.
Without getting the new permission you are not allowed to drive it on public road. If you still do then you might also have the problem that your insurance is void.
I don't know what e.g. TÜV would say about a device like this, but I guess the outcome is it would be either to expensive to analyze it or not possible it all since the overall system design and the interaction between the different devices might be not be known (e.g. there might be no public knowledge on how Acura ECUs might react on CAN signals which were never considered during design).
Another way to say this is, there's nothing dangerous about the hardware. You could be using it as a logging platform for the data on the CAN bus in your car. But letting the software send commands and take control from a driver is a different story.
IAANAL, but I don't think even that is true. I think it actually is legal to allow it to take control of your car, provided you are still sitting in the driver's seat and can take control back at any time.
Can you GUARANTEE that you would be able to do this in a modified car with such a setup? You might be able to do so if we are talking about pure mechanical overrides. With blackbox software that's imho next to impossible. There will be no public documentation on what the cars systems will do if the receive some commands on the bus from a non-approved device and others from the driver. Even if you can observe through testing that driver inputs win the system might still behave different in some edge cases.
but if you're going down the highway and the AI decides to suddenly swing the wheel far right as quickly as possible, no human being would have reaction time good enough to recover without crashing. I'd be pretty concerned if you could add an autopilot to your car and drive it around on public roads without any sort of demonstration of safety.
"The EPS controller in the car limits the torque to a very small amount, so regardless of the message, the controller cannot jerk the wheel" [1]
[1] https://github.com/commaai/openpilot/blob/master/SAFETY.md
I can put a Chevy engine in a Ford, and plenty of people do much more than that, e.g. homebrew EVs, etc.
wondering whether this can run on raspberryPi...
Big kudos to Geohot!
[1]: http://learnbonds.com/131150/tesla-ios-self-driving-comma-ai...
Imagine that a self-driving car detects a child in the road, and the only way to avoid hitting the child is to crash the car in to a wall, certainly killing the driver.
What choice should the car be programmed to make?
Who should decide what that choice is?
Who is ethically responsible for that choice?
In a normal car that is driven by a human, the choice is obviously made by that human and the responsibility is theirs. But with self-driving cars it's not so clear.
IIRC part of the explanation was that a human is unlikely to make a qualified choice in a situation like this anyway, so programming a decision matrix based on the utility of the target into the crash avoidance mechanism is moot. Something like 99.999% of the safety benefit would be achieved by a faster brake response time.
These systems are looking for obstacles and road contours. They don't know that the obstruction in the road is a human baby, let alone the composition of an adjacent wall.
(Ok, I used a "head on collision" number because I couldn't find a wall number, but the physics are probably pretty close. Numbers from a New Zealand Ministry of Transport PDF from 2012: http://www.transport.govt.nz/assets/Import/Documents/_versio... )
There is really no "right" answer in the absolute sense, but it's a great party trick to toss out there for a group of people to debate over.
Then we can adjust speed limits and move dumb walls and so on to reduce the deaths attributed to random selection by an autodrive system. Kind of like we do with deaths attributed to poor driving.
(I say if in the first paragraph because you need at least 4 conditions to be true: the information to make the choice is available, the vehicle is sophisticated enough to make the choice, there is enough time to make a maneuver and there is not enough time to make a maneuver that saves everybody)
From an engineering point of view you can just hit the brakes and never swerve. Good enough.
How much say would insurance companies that insure the automakers have? What about the government?
Also, what level of information about how this logic works is necessary to share with a potential buyer to judge them sufficiently informed?
I know I'd sure as hell want to know if there were scenarios in which my car would decide my life was not worth saving so I could do my best to never even come close to making one of those scenarios potentially possible.