Hacker News Reader

Comment by etaioinshrdlu | Hacker News Reader

Comma.ai represents a strange attitude towards life in which you try to turn hard, serious problems (like self-driving), into trivial problems (write some Python and run it on Android phones and plug it into your car!) and try get away with it. It only has a chance of working if you are cool enough in SV such as geohot. I interviewed there once and it was extremely off-putting how clear their demand for fast progress was, and how little they cared about safety.

It's move fast and break things all over again.

Think about how the tech community rips apart Boeing and demands the utmost in quality engineering, reliability, redundancy, testing. And then we have comma, which controls your vehicle on a non-realtime system...

If you ran comma.ai in your car and had a serious crash, you could possibly be found criminally negligent.

View on HN

> And then we have comma, which controls your vehicle on a non-realtime system...

Lol. This is what I found extremely shocking and is the first thing that I noticed. I don't think anyone serious about embedded design would touch it. They are trying to ignore decades of experience in real time systems and build some hacky stuff and make it open source.

I don't understand how you'd do hard real-time when one of the major inputs, the neural-networks, aren't deterministic? In a hard real-time system, what happens if your neural-networks are late telling you where the lanes are?

In reality the comma-AI isn't controlling the car, a number of peripherals on the canbus are controlling the car. All those peripherals are hard-real time, and the comma-AI is bridging the non-deterministic neural networks and those hard-real-time peripherals.

At some point you need to deal with the non-deterministic neural networks, and bridge them into a hard real-time control system. People complaining that part of the system isn't real-time just seems like they haven't thought about it very much. Of course the UI and neural networks aren't going to be hard real-time.

Neural networks are very much real-time. They generally consist of a fixed series of math operations of fixed size. Only the weights and inputs change.

The neural net will never be late. It just might be inaccurate.

Huh, I suppose they aren't inherently incompatible with real-time constraints. It looks like it's pretty challenging to do in practice though.

I'm still somewhat confused as to how multi-object recognition works in a real-time way, naively I'd presume that recognizing n+1 objects in a frame would take longer than n objects.

You might be able to more practically get real-time neural networks using something like YOLO (you only look once)?

Either way it seems to me like hard real-time neural networks are a very challenging problem.

> If you ran comma.ai in your car and had a serious crash, you could possibly be found criminally negligent.

I don't know that it's much different than using cruise-control and then having a serious crash. If there's criminal negligence when using cruise control, it's because you weren't aware-enough to shut it off when the conditions cruise-control is built for no longer applied; the mere fact of using cruise-control isn't criminally negligent, as long as you're "just waiting for it to slip up so you can take the wheel."

For that matter, the same is true of teaching a teenager to drive. There's no negligence to having them at the wheel if you're ready to supercede their bad driving at all times.

What's suddenly different, if the "thing that doesn't really know how to drive yet, but which you're ready to supercede the bad driving of" is an AI instead of a teenager?

You can get a learners permit to teach your teenager completely legally. You couldn't do the same with a 10 year old...

If a serious crash involving comma.ai went to trial, it would look like the driver was using unapproved, unregulated software, unnecessarily, putting others at risk. Recent Autopilot incidents show us that expecting a human to take over within a second is not reasonable. I would be very worried that the court would indeed find the driver negligent.

> I would be very worried that the court would indeed find the driver negligent.

I use open pilot on the highway, and I love it. I had to physically tap into the CAN bus for my car's safety system, that made me very nervous. But the installation is completely reversible, and I'm much more comfortable with it's limitations. I just treat it as a really great adaptive cruise control and active lane keep assist.

But that's all it is. I would be completely responsible in most situations if my car caused an accident while open pilot was active. Open pilot's design has it immediately drop all control of the car if you touch the brake or gas, it does not aggressively follow cars, and the driver is still required to control the car. So if I start to become uncomfortable I need to immediately take over.

Assuming that open pilot will always act within it's safety guidelines doesn't mean that its safe. Toyota's paid tens of millions in fines and settled multiple lawsuits over unintended acceleration. I believe that using openpilot on the highway means I'm a safer driver. But I do understand that if I did get in an accident proving I'm not negligent might be extremely difficult.

I agree with you, liability is a very important part of self driving functionality. I won't think a car has full self driving capability until I can get in the back of that car, go to sleep, have it drive me somewhere, and hold no legal or financial responsibility for any accidents that occur. Openpilot is nowhere close to that, but I do think it's really good for what it's meant to do.

What if the crash is caused by the device failing? In the case of your car's built in systems, they've been approved by regulators and you probably wouldn't be held liable. If it was an unregulated device that you installed into your own car that caused the crash, I bet the legal circumstances would be different.

Geohot is quite abrasive.

he's quite a character, surely abrasive to some. If you want a sample for yourself:

https://www.youtube.com/watch?v=u-dgbrB7PZU

His interview with Lex Fridman was quite interesting: https://youtu.be/iwcYp-XT7UI

I like him, personally.

Yeah I really enjoyed this interview. Made me respect him much more and realized that he actually is concerned about safety, specifically more than Tesla (especially when it comes to driver monitoring).

I’ve seen worse characters (Torvalds) but yet they are loved by the ‘community’ despite their behaviour.

The Bully wins

Compare OpenPilot to other solutions.

Tesla trying to kill someone: https://www.youtube.com/watch?v=0GnysB0rO3s

The super reliable "Level 3" Audi system not being able to go straight on a highway, the most simple problem there is in self-driving: https://www.youtube.com/watch?v=WsiUwq_M8lE&t=297

Several cars "pedestrian safety system" failing on detecting pedestrian (You had one job..) https://www.youtube.com/watch?v=7Y8JG7kepwc

Let's be honest, there is not a single driving assistant that isn't dangerous. Why care for "realtime" if your model tries to kill you once per minute? Realtime is one of the final touches you need to make your software perfect. That's just how the industry works, and if you want to keep up with them you have to go the same route, or do you know of any exception? Most manufacturers tell you that you should closely watch what your car is doing on autopilot, and they limit it's torque and acceleration, so the human driver is reliability, redundancy, testing and the realtime system.

Great perspective. Why isn't SDC software held to the same regard as Boeing? The safety framework is the same.

People don’t seem to understand probability and statistics.

And comma ai deals with regulation by selling dashcams that can drive a car if the user installs hardware and opensource software.

I use open pilot for highway driving and love it. But Comma AI is intentionally making design decisions to get around regulation.

Brakes on a plane, fall out of the sky Brakes in a car, come to a stop

Thanks to NHTSA being mean Comma is more or less or fore-er running(than Tesla they say...yeah okay?) so

I think their bet is to be acquired by a desperate established car manufacturer, hopefully before someone dies.

"And then we have comma, which controls your vehicle on a non-realtime system..."

This can't be true ... ?

The safety critical stuff for disengagements and torque limiting etc. runs in real-time in C code on an STM32 micro.

> It's move fast and break things all over again.

And this time it's people what they're moving :P

I'm curious about this perspective and would like to know more, could you expand on this?

Apps crash a lot.

IRL too, now.

If you'd like to know more specifically about the comma.ai saga, regarding safety and regulation in the U.S., here's some background, previously covered on HN:

https://news.ycombinator.com/item?id=12840368

https://techcrunch.com/2016/10/28/comma-ai-cancels-the-comma...

When did Comma.ai claim they were doing self-driving?

What are they doing?

Well, the website says:

> Currently, openpilot performs the functions of Adaptive Cruise Control (ACC) and Automated Lane Centering (ALC) for compatible vehicles. It performs similarly to Tesla Autopilot and GM Super Cruise. openpilot can steer, accelerate, and brake automatically for other vehicles within its lane.

So nowhere, then.

What is Adaptive Cruise Control and Automated Lane Centering if it isn't self driving?

Notably, the NHTSA called out these exact features in their letter to Comma.ai: https://techcrunch.com/2016/10/28/comma-ai-cancels-the-comma...

If the car can't drive without you, then it's not self-driving. Those features do not allow the the car to drive by itself. It can't go anywhere without a licensed, attentive driver. This is so thoroughly not complicated.

What hasn’t changed is that OpenPilot falls under the category of level 2 self-driving as defined by the Society of Automotive Engineers[1]

Look at the directory structure for the software - it has a "selfdrive" directory where all the code is![2]

[1] https://venturebeat.com/2020/01/07/comma-ai-launches-comma-t...

[2] https://github.com/commaai/openpilot#directory-structure

Reply on news.ycombinator.com