We Tested Comma AI's OpenPilot | Hacker News Reader

We Tested Comma AI's OpenPilot

thedrive.com

195 pointsclouddrover159 comments

The most surprising part about this to me is that there seems to be a somewhat standardized computer interface for controlling cars with an external device. I would never have guessed that, I thought for sure every maker would come up with their own unique design and require you to buy adapters or whatnot.

CAN bus is the standard physical interface. What is not standard is the message being passed, so each car will have it's own message for car information. What surprised me about OpenPilot is how some of the electric steering system can be tricked into turning using false sensor data. It's actually amazing that almost all cars now can be driven by software the only thing missing are more accurate and complicated sensors.

OpenPilot doesn't "trick" the steering system using "false sensor data". The steering controller just accepts messages for commanding steering over CAN bus. The same messages are used by the stock ADAS system.

Also, this bus is not usually exposed to the OBD port, so OpenPilot just connects to the same port the stock ADAS ECU uses.

OpenPilot pretends it's the ADAS and some cars you can pass the torque sensor on the custom serial bus used by the cars. There's not much difference in tricking or pretending or overwhelming the serial bus with another data message to make the steering system respond to OpenPilot.

If I recall correctly no car with standard harness openpilot support use an intercept configuration, only community supported cars.

The message it's sending is distinct from the torque sensor reading. There is a dedicated message ID for automated steering inputs, usually used for lane keeping assist and for jiggling the wheel as a departure warning.

I tried programming a module for my car without a lane assist ADAS system. I only stopped because it was my daily driver. I am still saving up for another car so I wont be afraid to break my car.

This is actually quite common with aftermarket tuning for the past decade. Traditional ECU tuning you replace your car's ECU with one with a modified (chipped) ECU. Now, there are bolt-ons that connect through the OBDII port. They modify shift points, change timings, and air fuel mix ratios. And do this without voiding the warranty.

[0] https://www.racechip.us/

Virtually all of these devices void warranty. The trick of course is to put the original ECU back in before any service where it would be noticed, however unlikely.

But that probably will void your MOT and your CO2 rating which in some places can get you in very hot water (or might lead to insurance not paying out in case of accident if the specs are an x% better than they were before the mod).

It is actually concerning that you can inject false sensor data this easily. I mean, sure, the messages on the CAN will not be encrypted or signed in any way, I know, but still...

Why exactly would be concerning? You need to physically plug cables in the car you yourself own.

If anything, every access to hardware you own SHOULD be this easy.

Well, not exactly. Once you have plugged this device into your can, it can be hacked. It's just a phone after all, and it has normal wireless connection options available.

It seems the system only works when the driver activates the cruise control, which hopefully Honda has programmed correctly: i.e. if cruise control is deactivated, steering inputs from the CAN-bus will be ignored.

So the driver can still regain control from a hacked device by deactivating cruise control, which s/he can do by moving the steering wheel or pressing one of the pedals.

The CAN bus is shared by a number of devices, each with its own microprocessor and thus (in principle) potentially subject to exploits that cause it to run malicious code. Some of the devices on the bus interact with the outside world via cellular, Wifi, Bluetooth and USB. The disturbing idea here is that your car is basically an internally-unfirewalled network of computers that can kill you if they malfunction.

But it is firewalled (between OBD port and the car, between internal buses,...). I won't say it's sufficiently locked down, but as we recently saw - not even Apple got that completely right - but there are several layers of protection(s) on the various communication bus-es and computers. At least on European Manufacturers.

I agree that this needs more hardening, but reading this comments one would get an idea that a 12 years old kid with a 10EUR dongle can hack any car. When in reality, stuff is a lot harder to do.

They can firewall or encrypt the bus, but if security fails, that can kill you as well by erroneously refusing to work.

Because that means not only you (the owner) can do it.

Also (caution, tangent): MCUs in cars is one of the systems where I want more encryption/signing and less hacking - especially for owners. Foremost to get rid of all the chip-tuned soot sources I am constanly driving behind.

The manufacturer knows how to control the engine, the random idiot with a laptop in 95% of the cases does more harm then good. If the manufacturer botched the MCU (hello VW), then he is held accountable. If he allows users to cause harm to the environment (chip-tuning), he should be held accountable too.

I really wish to know where you live that the quality of chip-tunin is so bad.

In developed world (EU), the owners are accountable for their vehicles. Including emission controls. But the problem is on the enforcing side, from the people actually doing the enforcing to the (un)avability of appropriate measuring devices that are able to detect the prescribed levels.

EDIT: As I saw that you are from AT. You do know that your country has a reputition of 'hardcore' car modification checking at the inspection? :D But yeah, I noticed several problems with that and your system of 'we will just let every mechanic certify the cars'.

There is no such thing as high quality chip tuning, unless you are replicating 100% what the manufacturer just disabled for your MCU (e.g. unlocking some kW your motor is in principle capable of outputting). But then again, e.g. BMW uses different, more reliable mechanic parts (camshaft, head, etc.) on motors that have these kWs unlocked, so you are still risking ruining your motor in the long run.

Chip tuning always meddles with curves the manufacturer chose after long consideration and extensive tests (let's just ignore VW for a moment) - and you are trying to tell me that guy from this shop just "knows better" and "sure, this will be better in every possible aspect"?

Are you trying to say a 2019 BMW out of factory will shoot out black smoke while accelerating (quite boringly) from a traffic light?

I think you are missing the point; perhaps there are bunch of kids just fiddling with their ECU on an otherwise stock car where you live.

In my experience though, tuning is mostly done in conjunction with changing other parts (headers, ports, cams, add a turbo, fuel kit etc.) so the manufacturers long consideration of the best balance for the stock engine is no longer relevant. You have to retune to get proper performance; fair that tuners are rarely considering emissions like the manufacturer would, but changing the airflow characteristics without changing the tune is a bad idea...

Let me try to explain:

Where I live, cars need certification/a license to be operated. The manufacturer does that for you, so you can obtain the required document quite easily for your car.

Aftermarket parts are regulated - you can't install what you want, it needs to be certified for your specific make and model (and motor variant, etc.) - otherwise your car is not legal anymore.

This applies to chip tuning as well. Since you're modifying a certified part, it looses the certification - obviously.

What new cars are allowed to emit, is strictly regulated.

Further, since this very month, we have a carbon-based tax on new cars (based upon the aforementioned certification from the manufacturer). Thus if the emissions of car are changed, you're basically evading taxes as well.

Where I live, chip tuning is almost only done to get a few HP on the cheap. And it is unfortunately very popular and nobody really cares.

Edit: You can seek in individual certification for some modifications, added to your documentation, but a tampered with ECU is not one of them I think.

Oh as I just remembered one thing that I forgot to add in the other reply - ofc I agree with most of your reasoning here. But I think that (regarding emissions) we need to find a better way to test/confirm this. As I believe you should be able to modify stuff you own.

But I just remembered regarding certification - BMW is selling official performance/tuning kits (Sometimes called M-Performance tuning kit), where you get a whole new ECU with more power (and maybe some other parts, but nothing major for sure). And now I really wonder how is the legality of that. And also adds another point, that there are valid and reasonably safe tuning options.

Ok, we live in different places and have different experience.

Most places I have lived in north america you are relatively free to install whatever after market parts you want, including things you have fabricated or modified yourself and people do some pretty extensive builds. That just doesn't work without changing the tune also.

I can guarantee you that there are some notable counter examples to your theory. I can tell you that manufacturers (BMW, VW,...) found out that it's a lot cheaper to manufacture the same physical engine and just limit it in software for different 'levels'.

BMW E90 series for example, from around 2008 onwards the 116d and 118d have exactly the same engine (per BMW part numbers mind you) but a different power output, even 120d only has one part difference (the ones with N47 engine that is). On VW part, new-er VW Caddy and VW Transporter with diesel engine option(s) have the same physical engine (in the model year) with widely different power outputs -> because they differentiate it for different segments via software.

And even moving from that, because of regional rules in NL for example, the cars with up to 140HP are prelevant and ofcourse VAG is not making a new engine just for NL, they limit one of their exsisting ones.

So yes, I believe that cars like this can be safely tuned without _any_ significant unwanted consequences for the longeviety of the engine/car. Mostly because the maps/factors can be adapted from their siblings with more factory power.

Regarding the black smoke/soot - no I believe that this is a (very) bad tune or something physicaly wrong with the car. It could be that they removed the DPF/EGR for some reason or something like that. But this is a whole other can of worms.

> Because that means not only you (the owner) can do it.

Not when it requires physical wires. This isn't some wireless TCP/IP stack connected to the internet.

Car entertainment/air conditioning systems usually also use a physically separate CAN bus so internet-connected dashboard devices don't get access to the engine and steering controls. They may share 12V power and that's it.

Comma.ai gets access to the "secret" CAN bus that can control steering via the lane keep camera next to the rear view mirror, not the OBD port or entertainment system.

To chime in (I'm not the original commenter you're replying to), in theory someone could develop an Internet-connected gadget, break into your car without being detected, plug said gadget in, and then drive you into a wall or off a cliff, maybe that's what the grandfather poster is worried about.

But in reality, no one's develop that sort of tech, and if a baddie wants to kidnap or kill, they'd probably spend their resources somewhere rather than hacking such a tool.

> Internet-connected gadget, break into your car without being detected, plug said gadget in, and then drive you into a wall or off a cliff

If someone wanted to make you drive off a cliff, they don't need a fancy internet-connected gadget. They could just loosen some screws on your steering column and leave.

If you have physical access to something you can always compromise it.

I would bet good money that there's many cars where the CAN is accessible by an ECU which has internet (or other remote) access.

Android and iOS auto, do you want these systems to attach this easily to the car?

Safety critical CAN bus messages should be signed.

Android and iOS auto don't have access to the CAN bus that can control steering. They may have access to a physically separate CAN bus that controls the entertainment system and air conditioning. There are normally multiple buses in a car.

Is there a legal requirement that the CAN buses be air-gapped?

What has changed since https://www.wired.com/2015/07/hackers-remotely-kill-jeep-hig...

Then you have to certify signing mechanism so that braking works even if cosmic ray ruined brake pedal control unit, and that is half sane and half stupid.

Not much of false sensor data, they use existing endpoints used by ADAS.

Steering or gas or brake ECUs on older cars without fancy adaptive automatic thingy don’t accept remote control, units in such cars only take analog user inputs and only report statuses on the bus.

Since ADAS is complicated and developed independent to the rest of the car, car manufacturers expanded those actuators to incorporate remote control mechanisms, and ADAS systems are implemented as a self contained computer that send out those control messages. Sort of like a Raspberry Pi with the camera in a case that is advertised as an “Ethernet AI camera”.

Openpilot on officially supported cars impersonate those ADAS unit. They are not directly meddling with pedal potentiometer readouts and such. That kind of rigging are used in some lab experiments but not in the majority of OP driven cars.

Yes I agree with you, I only call it false data because the ADAS sensor is replaced with OpenPilot. I tried OpenPilot with my car without ADAS and adaptive cruise control and you need the extra wire for the module to give false pedal position and trick the steering. I only stopped because I am afraid to break my daily driver, but it does work and it's amazing.

It's not that easy and it's getting harder with each new car platform. But on the other end, some of the sensors are basic and will always be suspectable to false inputs. But don't worry, maybe it looks easy -> but it's not.

It's not yet concerning since you need to hardwire to the custom serial bus of modern car to have access to steering and cruise control systems. What's concerning is the trend started by Tesla that remotely disables your car.

It would be quite concerning if they disabled cars based by “our automated systems detected breach of community rules” without ability to dispute it in future.

Right now disabling car after violent crash looks like obvious security feature that every car (especially electric).

It's mandated by regulation in all cars after..2002? It's a port most often situated close to the steering wheel, the interface is a bog standard CAN bus. Where it differs between cars is what command sets are used and allowed (eg sometimes proprietary ones are used, and controlling things may not be possible).

It's common to extract diagnostics about the car from this port. For example, when buying a used car this is highly recommended. Buy a cheap adapter that you plug in, connect to it over BT from a phone, and read out error codes. Perhaps the owner has temporarily put out an error to mask it during your inspection?

I personally respect OpenAI, Love the data link on cars. I think we should never be frightened of hacked cars, we should definitely be hacking our cars (and trucks, forklifts, lawnmowers, combines, papershredders, robots).

Just wanted to add many comments on Can and obdII:

In the United States the obdII port is required beginning in model year 1996. The obdII port and the CAN is analogous to rj45 and cat5, 5e, 6, etc.

Should note, the can bus is anything but bog standard.

However, in 2008 the United States required that all vehicles provide an iso15765 interface as the basis of the on board diagnostics. Iirc, 15765 is a subset of Can that's for cars and not forklifts. Can is controller area network.

There are a large variety of protocols talked across Can, far too many to list as you have noted. There are schemes which are serial, parralel addressed and not, and even analog signaling cohabitating this generic breakout port.

Where it differs between cars is the capabilities of the DLC, if the vehicles isn't equipped with a sensor suite, there will be no correspondent ecu subsystem to query, but for the most part you can send control messages to anything you want on a DoT approved vehicle, unless it is locked away behind a specific gateway and the car manufacturer is still hiding the method to control access to the device.

None of this applies to infotainment devices or comfort features.

Consumers should rue the day we return to the age of mystery subsystems requiring special protocols.

The reason I say this (and I suspect but, can't remember the 2008 iso 15765 mandate rationale) is that you probably want your mechanic to be able to fix your car, right?

Anybody who might be interested in the wild and fun world of automotive Com protocols such as this I can recommend to you this good and recent resource from 2020: https://www.csselectronics.com/screen/page/simple-intro-to-c...

Thanks for expanding on my knowledge. I have only experience implementing support for talking to a fluid property sensor in an industrial process and not much more. Thankfully, I might add.

Looked at it like a fixed phy layer with some parts of the messaging fixed as well, although now that you brought it up, I remember adding J1939 support too.

Ugh, it was, in my opinion, a bit-hacked mess :) I can really appreciate when things that despite being messy and hackish, solve real, actual problems, and for that, I thank thee, CAN-bus :).

This is a bit simplistic. By now, most cars will have multiple CAN buses, not least to separate the infotainment crap from the safety-critical subsystems like the ECU, steering, ABS braking and others. Chances are the mandated diagnostic part you can access will not be able to control these subsystems (or even receive their messages).

Of course a hardware air-gap is inconvenient when you are trying to ship subpar hardware and software on a shoestring budget, so the industry invented smart CAN "switches" and it's possible to escalate the access the diagnostic port provides, or tunnel from the infotainment network to the safety one. In newer cars, you can also possibly substitute CAN in this post with Ethernet.

It is mandated by law that cars have an OBDII port that supply diagnostic data over CAN interface for emissions testing.

It is not, however, required that said CAN bus is used for anything other than for passing emission tests. Easiest way to implement that feature is by exposing main CAN bus on that port, but you don’t have to.

> It's mandated by regulation in all cars after..2002?

I don't think the port is the surprising part, it's the ability to articulate brakes and steering wheel!

> Perhaps the owner has temporarily put out an error to mask it during your inspection?

To be clear, I don't have a car or a driver's licence, I was just surprised that this standardization was a thing at all. Seems like a stark contrast to all the walled gardens in every other technology-related industry these days.

Actually not really all cars have one standard. Most German cars use Flexray which Openpilot does not currently support and is a bit more tricky to hack into as described by Comma.ai: https://medium.com/@comma_ai/hacking-an-audi-performing-a-ma...

Only very recent cars use Flexray, and also not true for all manufacturers. CAN is still widely used, and some want to skip Flexray entirely for the next thing...

Only the really basic data (mostly related to emissions) and connector is standarised. Even the pinout of the connector (apart from the power supply and ground) differs quite a bit between manufacturers. The level of integration needed for actually driving the car is far from a standard. Not even that, most of the new cars have specific gateways which firewall the car electronics from the OBD port.

TL;DR: Apart from really basic data and physical connector, everything else is (ever more increasing) walled garden. With new VWs and Fiats requiring online connection for even most basic operations - so even if you know the protocol, you have to unlock it online (See VW SFD and Fiat SGW)

mandated for cars that have emissions requirements.

I think the obd connector on tesla doesn't do anything

Ha I never really thought of that. Now I wish I had looked for the port last time I was in one.

Teslas don’t have an OBD port.

some do.

the engine i'm not surprised, the wheel I am.. I thought powered steering was only amplifying, not fully controlling.

I recommend George Hotz's livestream [1] to anybody interested in computer science, really fascinating.

[1] : https://www.youtube.com/c/commaaiarchive/videos

https://www.youtube.com/watch?v=lPQHv3z3MWY

Guess I'm old... cuz that was painful to watch. Reminded me of the juvenile videos that Digg founders released back in their heyday. Sometimes you don't want/need to see behind the curtain!

His channel is recruiting tool for talent, not consumer facing PR. As such the content is very well suited.

George Hotz continues to keep blowing up the tech space somehow. I legitimately wonder if he's a savant, because he seems unstoppable. I did hear he has also stood on the shoulders of giants to get to where he is as well, so who knows.

Was this written by GPT-3? This is such a strange sentence

But yes George is definitely the misfit hacker wonder-kid type, and in both senses of the word hacker. He was one of the first to jailbreak the iPhone and the PS4 or 5 (can't remember which)

It was PS3

He streams on Youtube/Twitch fairly regularly, just from watching it is totally obvious that he is a genius-level programmer. The speed with which he reads and comprehends academic papers is incredibly impressive.

I watch and very much enjoy his Twitch streams, and he's definitely and obviously very intelligent, but I'd say not necessarily more than a top ~70th percentile HN commenter who works in software engineering. I don't think he's necessarily a genius, but just super curious and motivated and confident in his ability to learn anything if he dedicates enough time and practice.

You can see his learning and motivation process in his streams, when he dives into something he's totally unfamiliar with. It's very inspiring and fun to follow along with (partly because he clearly is having fun). He really enjoys learning things and has the kind of ego (not in a bad way) that just doesn't let anything stop him from trying to catch up on any concept in any field, even if he's totally "veering outside his lane" and going in blind (like his COVID-19 bioscience streams). He has a deep, almost hypomanic hunger for knowledge, and keeps Googling things until he understands everything he's reading. I think a lot of it's psychological and not purely IQ/cognitive speed, though of course it helps.

If anyone's curious, you can find an archive of his streams here: https://www.youtube.com/c/commaaiarchive

Hotz won CSAW singlehanded, a one-man team against multi-person teams. And he won by an enormous margin. He is clearly way above the 70th percentile.

You're right, I shouldn't downplay his abilities, and he's definitely well over 99th percentile when it comes to results and accomplishments at the very least. I've competed in CSAW and some other CTFs and winning as a one-person team is definitely a massive achievement.

Just to counter a tiny bit, he didn't necessarily win by an enormous margin. CMU's PPP has overwhelmingly dominated every collegiate CTF since forever, and he was a CMU student and I wouldn't be surprised if he trained closely with PPP members. He got 5100 to PPP's 4800, which, to play Devil's advocate, depending on the particular point weights of the solved challenges, could potentially mean he solved one additional challenge over PPP (though possibly more; I didn't dig to find the exact breakdowns). I believe he also deeply specialized in low-level RE from a young age (as demonstrated by the iPhone and PS3 exploits), and from my perception that was less common among collegiate CTF competitors and gives you a huge leg up. But of course mastering something that difficult so young and being at CMU are itself huge achievements, as is outmatching PPP on anything at all entirely by yourself.

Obviously insanely impressive, and exponentially better than I could've done (and did do) at that age or could do today. Be it intelligence or endless drive and work ethic or all of the above, he's clearly an extremely skilled polymath. I guess I just didn't want people feeling too demotivated over genetic factors, which play a big but not a full role. He may very well be a genius, though.

With every halfway decent engineer you personally know, go compete in a major competition he won by himself in his early 20s and then repeat your statement. The archives are all online, take your time.

I'm probably biased because I competed in those competitions in college and do know some people who did very well (though of course not as well as him). He's definitely a > 99th percentile RE and hacker, to be sure. I was just talking about raw cognitive ability, but he absolutely may very well be > 99th percentile there as well.

Back in the early days of csaw it was easy as hell. There wasnt a lot of competition.

We are all standing on the shoulders of giants to be honest.

Very nicely executed device with a good price point. Of course it's an utter nightmare with regards to privacy and I'd be very nervous regarding security and safety when I'd hand over the controls of my car to basically an android phone. Maybe I skimmed over it but I'd be interested in knowing which phones they use exactly.

So I applaud the engineering but it's not a device I'd buy (and yes I do own and carry around a cell phone).

My main concern would be that the architecture of an OTS Android system does not offer any guarantees regarding real-time i.e. hard deadlines.

Ever had an android app or even the whole phone locking up? Even if it is only for a second or two, I don't want this to happen to a device that feeds live data into my car's controller.

Even if this is only level 2 driver assist. If a car would get a bogus "you are crossing a divider line" signal for e.g. 2 seconds, it would start to steer quite persistently (even if with limited force) in a certain direction and you would have to counter that action.

Nope, I wouldn't use this.

I think thats the ultimate plan, but if you listen to GeoHotz talk about it, he really wants an economical self driving solution. E.g. waymo cars cost 100k+ and thus isn't economical.

It's also opensource so I think the end game is to have manufactures adopt this and go through the whole song and dance of getting it full validated on their vehicles.

Also likely selling access to the ml model and/or training data. Openpilot is accruing many trips/miles for their dataset, the model and data isn't open-source.

That appears to be where the gold is stashed to my eyes.

If the software is open source and works very well, comma will still have a long lead in developing a great self driving model for which the bricks are miles driven down the road in normal commutes and trips by real drivers in real world conditions.

AutoPilot disengages if you try to take over the wheel, it's one of the basic features that is available.

The only instance where you are steering is on the new lane switch (turn on blinker, check lane, depart from current lane)

Are you talking about Tesla AutoPilot?

This system is meant for cars without level 2 driver assist, so not for Teslas.

Further, the system has no control over what "disengages" in the car itself - it is sending bogus sensor inputs to elicit a reaction according to level 2 driver assistance. Thus, if it sends such bogus inputs and fails at doing this "reliably", it will result in misbehaving steering, breaking, etc. which it does not control itself.

Further, if it was in control, my argument would stay: If your controlling device has left the building (crashed), it can also not disengage anything any more reliably.

Openpilot uses an embedded controller that is connected to the phone via usb to control communication on the CAN busses. It is what enforces the disengagement, and the code on it follows the MISRA standard. The phone can send random commands and you should still be able to maintain control of the vehicle.

Check out the safety architecture section. https://medium.com/@comma_ai/how-to-write-a-car-port-for-ope...

The driving model runs on the modified android OS (NEOS), but the safety critical code runs separately real-time on a SIL2 STM32 microcontroler. Comma strives for ISO26262 compliance.

The next comma hardware will ditch android and phones though.

I'm much less worried about privacy with an open-source solution. At least I can inspect the various ways in which they spy on me, and if I'm super extra motivated I could stop that. Tesla, on the other hand, is the real nightmare.

Can I even buy a new car in the U.S. these days that isn't a rolling Orwellian nightmare?

Honda's cars (which are compatible with OpenPilot) don't connect to the internet unless you really want software updates, and those can be transferred via USB easily (or you can just not update it, since it's a car).

According to the article it is a LeEco Le Pro 3

The android phone they use for the comma two is a LeEco Le Pro 3.

The huge video data set that their users are sharing with them could become a valuable asset for Comma AI.

They actually already make it available for anyone to buy. $250k for first month and 100k a month after. No one has bought the service yet apparently.

The project contains software that acts as functional safety software in a vehicle, so ISO 26262 should apply. I couldn't find any reference to this in the repository or landing page?

Some sources mention it is for research only purposes (in closed roads?), others advertise this as a CES consumer product, has the project evolved along the way?

Each car manufacturer has to provide a battery of tests to regulators for each vehicle and sub-system within each vehicle. This project claims to be safe to use on more than 100 separate vehicles, perhaps I am missing something but this claim would require a considerable amount of evidence.

I am all for open-source and experimentation but in cases where we cannot provide guarantees, this should be done on a private road at your own risk.

Afaik you buy this thing as a dashcam and then flash the self driving software on your own at your own risk. This was done exactly when "the National Highway Traffic Safety Administration (NHTSA) demanded that Comma.ai provide proof to regulators that its proposed device for self-driving cars would be safe". [1] I wonder if there's anything new around this development with regards to Comma.ai as this is status of 2016.

[1] https://www.reuters.com/article/selfdriving-safety/comma-ai-...

> on your own at your own risk

It is not just your own risk but also that of other pedestrians, cyclists, and drivers that share the road with you.

that's not what "at your own risk" means.

"At your own risk" means you bear the risk (legally). If something happens, it is on you, and the manufacturer offloads any liability on you.

That does not imply what the risks are. Life threatening? You are risking other people's lives.

The legal risk should scare people. If I reflash my dash cam, then I’m the only liable party. The insurance minimums in most states are criminally low. Whereas if I’m using something built into my car, the manufacturer’s deep pockets should at least deflect most of the liability away from me.

No need to be pedantic. No-one's arguing about the meaning of 'at your own risk', it's just that your decision to use this software affects more than just yourself.

I think the response you are replying to was a reaction to the possible interpretation that OpenPilot puts pedestrians etc. at increased risk, or that it put the driver at increased risk.

While a reasonable interpretation includes putting the driver at increased /legal/ risk relative to a system where the manufacturer represents it as for sale for autonomous driving, it is unreasonable to infer that this puts the driver at greater risk of being sued, having to sue, or monetary loss. OpenPilot may be better / safer than the competition such that using it lowers your risk of accident, monetary loss, etc. even including any purported loss of the ability to pass the buck to the manufacturer.

To be fair the person they're replying to was the one who was either being pedantic of trying to score some cheap virtue points by stating the obvious (and I'm giving them the benefit of the doubt and assuming the former)

Don't you just love cars. I'm glad they gave us so much independence.

I'm not sure where you looked, but https://github.com/commaai/openpilot states it very clearly: "openpilot observes ISO26262 guidelines, see SAFETY.md for more detail."

Thanks jmiskovic, I stand corrected:

> openpilot is developed in good faith to be compliant with FMVSS requirements and to follow industry standards of safety for Level 2 Driver Assistance Systems. In particular, we observe ISO26262 guidelines, including those from pertinent documents released by NHTSA. In addition, we impose strict coding guidelines (like MISRA C : 2012) on parts of openpilot that are safety relevant. We also perform software-in-the-loop, hardware-in-the-loop and in-vehicle tests before each software release. [0]

I would be quite curious to see how they perform HIL tests and in-vehicle tests and if those results are also public?

[0] https://github.com/commaai/openpilot/blob/master/SAFETY.md

Edit: I cannot edit or remove the grandparent comment but I leave here this correction

Yes, but what does "observes" ISO26262 mean? The words I would want to hear before plugging this into my car are, "meets", "complies with with", or "certified to ISO26262."

They run Python codes on the phone, and let a separate microcontroller do car interfacing, in which they do their best to gatekeep the ML output. That “critical” uC part is self certified by them on best effort basis.

And of course that uC unit comes in a 3D printed enclosure so yeah

There is ofcourse no way they are ISO26262 compliant since they are not even trying to make it road legal.

ISO26262 compliance is really heavy and requires enormous amount of documentation and requirements tracking.

They plan on open sourcing their ISO26262 documentation when Openpilot 1.0 is released. They were hiring (or already did? don't know) someone to help them write it up for release.

I am willing to bet that the nimble AutoPilot selling <$1000 hardware and $25/month subscriptions will outlive the billion dollar giants like Cruise. CommaAI is small but it is a profitable company that controls its destiny.

Waymo and Tesla have some hope. Cruise and Zoox are burning VC money like crazy with little to show.

> Waymo and Tesla have some hope. Cruise and Zoox are burning VC money like crazy with little to show.

The numbers for Cruise reported by GM [1]:

As of six months Q22020, Cruise posted revenues of $53m and total expenses of $498m, compared to $50m in rev and $553m in expenses for the same six month period in 2019.

Total cash in hand for Cruise is $2.2b with zero debt

2 more years of cash burn is not your typical startup that is 'burning VC money like crazy', but still the cash burn is large.

Now the question is if Cruise can expect any revenue from the new DMV authorization in SF [2]

[1] starts page 9 https://media.gm.com/content/dam/Media/gmcom/investor/2020/j...

[2] https://www.dmv.ca.gov/portal/news-and-media/dmv-authorizes-...

Zoox is burning Amazon's money now.

MobilEye has been selling ADAS kits for about a decade now. I’m not entirely sure why CommaAI is a special case here.

Things that are that price point end up just getting acquired by auto manufacturers since the tech isn't used anywhere else, and they can run the price down by owning it and producing it at scale / having some competitive advantage temporarily.

It's neat, and a good application of the technology, but honestly I don't see this taking off because the gap between the tech and execution is closing quickly. Comma was really far ahead of everyone 5 years ago. But... we have gone from a situation where the Level 2 systems available in most new cars went from an impractical gimmick, to very useful and comparable to this in just a few years. New Toyota's (MY 2020) are pretty comparable to this at no extra cost. It's really hard to see why you would buy this system for $1100 if you can just buy a new car and get this all for free, plus a bunch of additional safety features that OpenPilot obviously can't include: backup sensors, blind spot monitors, etc.

I don't think manufacturers will outsource the development of their ADAS systems to Comma (or anyone else for that matter) because they view it as a potential short-term differentiator. The existing differentiators are collapsing quickly: reliability is no longer a major concern for EVs, infotainment is lost cause (Carplay and Android Auto won decisively). Aside from ADAS, we're looking at a scenario where in 10 years the only reason to buy from company X vs company Y might be price.

I'm glad you are happy with your new Toyota but I think you are giving too much confidence in it's ability. Look at this evaluation between OpenPilot and Toyota TSS 2.0 https://youtu.be/z5-inxH92wM

Also free to checkout navigating SF with OpenPilot https://youtu.be/0TpMMoQ7GGg (30 min). I highly doubt you would find similar results with a Toyota TSS 2.0 system. Nevertheless, I will let the contents of the videos speak for themselves.

I have a Subaru with Eyesight, not a Toyota. But my point was more so that one of the most popular car manufacturers is giving away a system that is probably 70% as good as OpenPilot.

Those videos didn't really change my mind. I'm not convinced at all that it's really that much better on the highway. It's definitely better for non-highway driving, but it also had a few disengagements that he had to intervene for.

Look, if it was $1000 for Level 5 autonomy I would be the first in line to purchase it. But it's clearly not anywhere close to that yet.

I have a 2019 Subaru with Eyesight. The system isn't half as good as you think it is. The comma system is seriously so far ahead in regards to LKAS (lane centering, cornering, etc) than Subaru's eyesight system.

They never said Subaru's system was as good as Comma's. They were talking about Toyota's.

It was a bit confusing because they don't own a Toyota, but they do own a Subaru.

> New Toyota's (MY 2020)

> I have a Subaru with Eyesight, not a Toyota.

I'm guessing MY = Model Year?

I did buy a new car a few months ago, and the best possible driver assist features were a priority for me. I bought Chrysler Pacifica minivan with their "advanced" driver assist package. It's just an adaptive cruise control plus warnings when I drift off out of a clearly marked lane. It does not offer a lane keeping ability.

That's a $50k 2020 model with all premium features added. I haven't tried Comma, but from what I can see it can actually drive itself in the most routine situation - staying in a lane on a freeway. My Pacifica can't do that, even though it has 8 cameras and a radar.

This is a legitimate criticism. I can't for the life of me, figure out why manufacturers are still artificially segmenting their products with software. Some cars have had fairly capable lane tracing since the 2019 MY, and others still don't for the 2021 MY.

However, now that I have the adaptive cruise control, I get anxious that I might get into an accident when I think it's enabled while it isn't. Another concern is that it won't be able to apply braking hard enough when a collision is imminent - in such situation it flashes a red warning on the display and sounds an alarm prompting me to brake, but because I haven't been actively adjusting speed with my foot, it might increase my reaction time.

So perhaps having more of those "half way there" self driving features could lead to more of such anxiety, I'm not sure. And I don't know who I would trust more, a car manufacturer, or someone like Comma AI, with my safety. Leaning towards car manufacturers to be honest.

At least personally (as someone driving openpilot pretty regularly) I'm a lot happier to trust openpilot than the stock system in my car since openpilot watches the driver and nags you (and eventually disengages) if you don't pay attention to the road. With the stock system I have to trust myself more.

At least with manufacturers you know who to sue if something goes wrong. I'm pretty sure that Comma has set this all up to shed as much liability as possible since they make you flash the unit with the right software.

Yeah, the stock driver assists in most cars still need a LOT of work. I drove from San Diego to LA last weekend with a comma, and openpilot handled the entire trip without a single disengagement until I got off the highway. I’d be really impressed to see a stock LKA manage the same (although I’ve heard great things about super cruise!)

It's really hard to see why you would buy this system for $1100 if you can just buy a new car and get this all for free

A key phrase here is "just buy a new car". You might be able to afford that but most can't.

I don't know what Comma AI's market strategy is but I would imagine there is a huge potential aftermarket for gizmos like this in capable older motors, with that growing as time moves on and the enabling technology takes hold.

The only manufacturer who is doing better than Comma is Tesla. Every single manufacturer (including Audi and Cadillac) are years behind Comma in terms of performance.

Is that true though? I know that Hyundai at least has demonstrated a true level 4 system in Korea. It hasn't made its way to any production vehicles yet, but I suspect that's partially because the regulatory regime is still up in the air. Part of the real risk with releasing something right now is that you'll end up with a system that is not compliant with whatever the regulations eventually require.

Then why don't manufacturer just use openpilot ? afaik its free and the code is in the github.

Liability maybe? The same reason why everything except for TACC with Tesla is considered beta and you have to consent to always be ready to intervene to use it.

I guess the trained models themselves are not free,

But George is selling it fir the same price to them as well, they just don’t want to rely on him.

I'm not sure about that. You have to have a relatively recent vehicle for this to work, because steer-by-wire only became common place in the last few years as manufacturers started adding ADAS systems. This wont work on any car with hydraulic steering, which rules out a lot of cars.

The average car in the US is 11 years old.

Interned for Comma AI. The tech is indeed chill.

Did you happen to enable ghostride a whip [1] mode at any time?

[1]https://en.m.wikipedia.org/wiki/Ghost_riding

What is their payrate?

$1500/week

How long before we start seeing traditional car manufacturers just incorporating this into their vehicles? Why develop your own ADAS software when Comma.AI and OpenPilot can just plugin into their hardware?

It would have to be much cheaper. I would guess a front camera from a Tier-1 supplier will cost the manufacturer some millions in development (fitting in physically, UI, and protocols) and then a few hundred per device once. Tier-1s optimize for every cent for mature features like ACC as the margins are low.

The article reads that OpenPilot uses facial recognition to detect a distracted driver.....Can that be fooled with an image, or perhaps a mannequin? I wonder if you could apply this autonomous car tech to vehicle-borne IEDs. The driver could dismount a few hundred meters short of the target and provide eyes-on ground-level surveillance (if for some reason you aren't using a UAV for the same) while the vehicle drives on a final attack vector.

It's an open system. You don't have to fool it in that case. Just disable the distraction detection.

They probably also check for movements, blinks, etc. In any case, it should be viewed as an help for the driver (beep when falling asleep), rather than something working against him. If you wan to actively fool the device, do as you please (the software is open-source, after all, you can probably fork it to disable that feature) but face the consequences: your life is on the line!

I want a device that will not control the car but will give more a ton of real-time information, ideally projected on some corner of my windshield.

Information including:

1. Speed limit for the area/road I am in.

2. Real time radar report telling me how many cars/pedestrians e.t.c are around me.

3. Information on common accident types in the area I am driving in.

4. For my corner of the world, it will be good to get a report on road status before I take the route - something like gravel, cratered e.t.c

I suspect that something like this would sell like crazy for existing cars. Basically a thing were I install a camera/sensor that is hooked to such a device.

> 1. Speed limit for the area/road I am in.

That one already exists, many manufacturers already provide 'intelligent speed' systems, which combine gps and traffic sign recognition camera info to determine the current speed limit in that stretch of road and display it on your dashboard.

Yeah I think Satnav answers most. I guess the biggest thing we are missing is Lidar hooked to Satnav

Most satnavs have it already.

Partially...

- Satnav, is based on GPS which has a degree of error, it might place on a 4 lane road or a 2 lane road immediately parallel to it, with a drastically different speed limit.

- Satnav has no way of adapting to temporary lane closures or roadwork speed limits.

That's the reasoning for combining GPS with TSR camera data, in these 'intelligent speed' ADAS systems.

Satnav has no way of adapting to temporary lane closures or roadwork speed limits.

Waze is pretty good at this, presumably because the road/map data is crowdsourced

> 1. Speed limit for the area/road I am in.

My 2018 Honda Accord uses the camera to read speed limit signs and displays them in both the heads up display and in the middle of the gauge cluster.

Then I use Google Maps with CarPlay to offset some of that other information. It will alert if an alternate route becomes faster, or a wreck is ahead. I've also found it's been pretty good to know about road closures as well.

>3. Information on common accident types in the area I am driving in.

I think this one is incredibly important. Although drivers/AI need to maintain an adequate level of preparedness for general situations... cities and police departments have historical data for accidents. There should be notifications about specific things to look out for. Net-net this might actually reduce accidents

sligthly similar in spirit.. I so wish I could have local information about stop lights..

I like to play the 'continuous driving' game where I coast in neutral if the next light is red and try to get the green light so save kinetic energy etc

With this data I could compute the avg speed required to do so (granted there are other factors because I don't want to clog the traffic too etc etc)

I haven't kept up with this space at all. I'm a bit surprised to learn that all that's required is a windshield attachment & software update. I find this simultaneously fascinating & scary.

I wonder how regulators and most insurers treat those.

It won’t surprise me if these might not be “road legal” modifications, or at least modifications that may incur additional criminal liability Im also pretty sure that insurers can use these to deny claims if accidents happen.

ive been looking at getting this for my CRV for about 1 year. most insurance companies are fine with adding this device to your plan as ive inquired and read about several experiences with it.

I wonder if they think it’s the same as aftermarket ADAS systems like MobilEye while it isn’t since it requires you to essentially flash your car.

I can see this influencing my next vehicle purchase.

For those who are familiar with the project - what is the security situation like?

I run mine completely offline, it doesn't require an internet connection except every few months. This check can be disabled pretty easily.

It requires a weekly connection, which they implemented so that if there were a bug that were in an upgrade they could deploy it reasonably quickly. But as you say, it can be disabled, because it's open source.

See also Ghost Locomotion, L3 device with more funding.

$3,495 Separate $99/month subscription required.

and according to their website....

> Coming soon

> The delivery and use of Ghost products will depend on our ability to meet certain reliability and performance levels, as well as receiving applicable regulatory approvals, which could take longer for certain car models and in certain jurisdictions. We continuously work to improve the functionality of our products, and as a result, some of the features depicted herein may be delivered to you via over-the-air software updates or equipment upgrades.

There's nothing to see there.

Its a scam?

It is clearly labeled "level 2" so calling that "autonomy" is misleading.

L4/L5 or crash. Anything below is dangerous because of the propensity for humans to rely on what seems to be working the majority of the time and artificially trust it, especially with a population trained to need to check their phones. We saw this with the infamous Uber training driver crash in AZ, we saw this with the Apple employee in the Tesla, and we'll continue to see this over and over again until we can finally remove the steering wheel so there's no possible ill-informed release of control. Naming L2 systems 'Autopilot' and a bunch of misleading marketing certainly doesn't help either.

Nonsense. Airplanes can do 99.9% of their flying without human intervention, but we don't hear about planes slamming into mountain sides because a pilot was texting or browsing the internet.

The problem is any idiot can get a drivers license that let's them pilot around a 5000lb steel cube for about $200 and 20 hours of training time. And punishments are extremely lax for when mistakes are made.

>Nonsense

> The problem is any idiot can get a drivers license that let's them pilot around a 5000lb steel cube

I don't see how it's nonsense, given that the driver's license problem isn't going to be solved any time soon. The problem is that our society practically requires basically everyone to be able to drive to function, unless you happen to live in one of the relatively rare places with good public transportation.

Flying in the air above the clouds and mountains is nothing like driving on a road with pure unpredictability all around you and needing to solve close to AGI in order to achieve it.

One major differentiator for openpilot is the active driver monitoring. If you keep checking your phone while it's activated it will disengage and eventually even refuse to engage. Other systems just make you nudge the wheel, but openpilot uses the front facing camera to watch your eyes.

Reply on news.ycombinator.com