Hacker News Reader

Comment by nytesky | Hacker News Reader

Did any of you read the article? She acknowledged there are some cases where it’s working in well defined environments.

But she clearly shows how we are currently in the most dangerous phase “car drives except handing off to human drivers in a crisis”.

“ The policy should be that either the computer is driving or you are driving. And by driving I mean steering—people do fine with regular cruise control. The act of keeping your hands on the wheel and guiding the car’s lateral motion is enough to keep your brain engaged. So, no L3 [full self-driving, but the driver must be ready to take the wheel], which is too confusing, and no hands-free L2 [partial self-driving]. I am not against the passing of control per se, but there should just be two modes of operation, with crystal clear feedback about which mode you are in.”

FSD has been 5 years away for… 15 years now?

View on HN

Fun story, I was in a loaner car from a dealership which had lane assist. I was driving in the desert around a corner against a cliff so I couldn't see around it very well. Right around the corner there was a small boulder in the road which I had to swerve to avoid, except the lane assist fought me on it since it would cross the painted lines. It wasn't like a near death experience or anything, but it really made me think on how the systems meant to protect us can be what causes a life threatening problem if they aren't designed well. If I had to swerve to avoid another rock or a wayward vehicle, compensating for my own vehicle's unexpected response to driver input is at the bottom of the list of things I'd want.

An extreme example of this is the Boeing 737 MAX.

If I recall correctly it was trying to emulate the handling of the 737 so that the pilots didn't need to go thrpugh lengthy training

Even without any obstacles a rented jeep Cherokee felt like it was constantly trying to kill me on mountain roads. The lane assist also turned itself back on every time the car started.

Another baffling feature was that the car refused to move after stopping at stop sign. Turns out there’s a “safety feature” to disable the engine if the seatbelt is not buckled. Thankfully I was just looking for parking and the only consequence was some inconvenienced cars behind me but that could get really bad in the wrong situation.

On my dad's car, the middle back seatbelt has a pretty odd behaviour. It doesn't stretch at all if the car is moving, but it is allowed to contract. This means:

- if the person sitting in the middle has failed to buckle up or at least stretch the belt enough before the car started moving, now they can't do it until the car stops again;

- whenever the person in that place breathes out, they're running the risk of not being able to breathe back in next.

- when the car accelerates, there is a tendency for the person to weigh back into the seat. That causes the belt to contract.

Repeat the 2 latter points enough, and it becomes practically impossible to breathe or even to undo the buckle so you get trapped quite literally dying of asphyxia and in pain from the belt pressing hard against you.

Maybe I’m missing something, but seems like your dad might want to take the car in for service. Malfunctioning safety equipment is a weird thing to let persist like this.

The bit you're missing is that it's working as designed.

What makes you say that?

* The fact that it's been through many full reviews

* The fact that other cars of the same make and model do the same

* The fact that (at least) some other models of the same make do the same

* The fact that apparently I was the only person surprised at it when I mentioned to a few people who have cars of other makes as well

* The fact that it was like this when it was new

Perhaps none of these in isolation mean it's as designed but I reckon taken together they're a very good indication that it is as designed.

I recall seeing a video of a car who managed to drive in a blocked zone and stop on rails, and eventually got hit by a train as it was unable to drive out - arguably because the driver opened the door, and afterwards the car automatically switched to parking or something.

About a year ago, was driving on a mountain road, the lane markets were eroded and the auto-pilot / driver assist thing stopped turning, If I wasn't paying attention it was straight into the barriers at speed.

I just prefer to drive, I actually don't mind driving either.

Had the same experience while driving through Tahoe in a Toyota Camry rental. Immediately turned that "feature" off because the last thing I want is my car fighting me if I have to make a split second decision.

Lane assist is a product of the pervasive "what works in urban San Francisco/other coastal cities is the right solution for everyone" mentality. Unexpected obstacles are far from the worst of its problems on rural roads. Lane assist doesn't give a fig about the fully loaded dump truck coming around the inside of the curve at a relative speed of 100mph within inches of your door, because that situation just doesn't exist in urban environments. (You are supposed to be all over your white line in this case, in case it's not obvious). Conversely, headlight tech which matters much more in rural environments (oncoming headlights are too dazzling on dark roads, and your own headlights are too dim - current legal tech fails both ways simultaneously) is irrelevant in urban environments, so it is literally illegal to improve.

Shit like this makes you realize how behind the curve NHTSA and other government regulators are. Why aren't they putting more guardrails around self driving? What do they stand to lose by enforcing common sense ideas like the ones in the article?

What do they stand to lose? Cushy industry jobs.

Because the USA is a country ruled by large corporations and/or wealthy individuals, not by the people. The large corporations wants to be free to do self driving experiments, so they do it.

They person interviewed, Missy Cummings, was appointed by the Biden administration to be a senior safety advisor to NHTSA. Elon Musk directed his fans (who are also largely shareholders) to viciously attack her which included sending death threats [1]. She was then ordered to recuse herself from any Tesla related matters due to a petition driven by those same Tesla shareholders.

[1] https://amp.cnn.com/cnn/2021/10/28/cars/tesla-ntsb-cummings/...

> Elon Musk directed his fans

Where'd he do this? Have a link? Curious.

As stated in the article, he called her extremely biased against Tesla: https://twitter.com/elonmusk/status/1450653942938054656?s=20

Out of context, the tweet is largely innocuous as plenty of people might tweet such things. However, immediately following that tweet Missy Cummings was harassed ferociously by the Tesla shareholder base as seen here [1].

One of the leaders of the petition immediately following the tweet was Omar Qazi of WholeMarsBlog who is such a rabid fan and shareholder that he sees no issue with endangering children to support Tesla’s stock price as seen here [2] and the corresponding article. He also advocates for harassing any critics with vicious, baseless accusations [3].

Even if we assume that Elon is not directly organizing the attacks or by proxy, he has deliberately shaped a shareholding fanbase who will at the slightest mention. And, even if we assume that such a virulent base was purely accidental, which it never is, public figures have a moral duty to consider the likely consequences of their actions or speech.

His followers, many of which he engages with and responds to on Twitter such as the aforementioned Onar Qazi, are well known to viciously attack people he identifies as critics. These are not fringe elements, but the core of his fanbase. His actions are all but guaranteed to result in harassment of his critics, which is to his benefit, and he makes no effort to stop such interpretation, but rather harnesses it to suppress his opponents.

[1] https://twitter.com/MoodyHikmet/status/1451332679597625344

[2] https://twitter.com/WholeMarsBlog/status/1558122848601812993

[3] https://dawnproject.com/what-is-the-credibility-and-characte...

You conveniently forgot to mention several things that don't fit your narrative. Why are you so eager to place the blame on Tesla fans for unearthing evidence, instead of calling out Cummings for her unsuitability for the role? Are you actually suggesting that NHTSA requested her recusal based on a simple petition, rather than the hard-hitting evidence it put forth?

Cummings had a seat on the board of a LIDAR company and was paid by them [1].

She has had research funding from competitors in the industry Toyota, Uber, and Volkswagen [2].

She had associations with Tesla short-sellers, which raises questions about her impartiality [1].

Notably, she made public remarks suggesting an inclination to punch Musk [1].

Moreover, her legal team tried to suppress public discourse by attempting to get the petition on Change.org removed [1].

Also, should someone like this really hold an impartial position like the senior advisor of NHTSA? [3][4].

Let's not overlook these potential conflicts of interest just because they are inconvenient to acknowledge.

[1] https://wholemars.net/2021/11/21/missy-cummings-is-fighting-... [2] https://twitter.com/wholemarsblog/status/1450710771399200769 [3] https://twitter.com/heydave7/status/1450691154299731970 [4] https://twitter.com/TitterDaily/status/1450832510267629570

Hah, tl;dr, but link 1 is sooo neutral, it describes with flowery language about how great Tesla self driving is, and how shitty LIDAR is, after going on attack about her investment in a LIDAR company...

You mean Veoneer, the LIDAR company where she was formerly on the board of directors and formerly a shareholder [1]? Where she resigned from the board on October 25, effective November 1, and divested all of her shares prior to accepting the appointment to remove the potential financial conflict of interest?

It sure is convenient how the blog post by Omar Qazi, a well known shareholder and propagandist who literally spends all of his time marketing for Tesla, made on November 21 fails to mention that event that occurred on October 25. Almost as if he has a massive conflict of interest and will say and do anything to support the value of his Tesla stock, such as endangering children [2]. I can not think of a more unreliable source other than Elon Musk himself (Elon Musk being a more unreliable source due to his even bigger conflict of interest).

And yes, a expert with decades of experience in safety engineering having a informed opinion supported by actual research they have done [3] on products is exactly someone who should have a position like senior advisor to NHTSA [4].

[1] https://www.nasdaq.com/articles/veoneer-says-cummings-to-res...

[2] https://twitter.com/WholeMarsBlog/status/1558122848601812993

[3] http://hal.pratt.duke.edu/sites/hal.pratt.duke.edu/files/u39...

[4] https://www.curbed.com/2021/10/elon-musk-missy-cummings-tran...

Fair enough, I'll concede that she did resign from the board and sold her shares. However, you've sidestepped the remaining five points in my argument. If these don't raise red flags for you about her suitability for an impartial role, then perhaps we fundamentally differ in our understanding of what's tolerable and what's not.

I think most people would agree that the middle zone is not ideal for most drivers.

However, there may be no other way to someday reach level 5 for general driving, other than to live with that middle zone for a while (keeping in mind that it’s already safer than humans driving alone) in order to conduct the training needed to reach level 5.

You don’t just wish, or regulate, level 5 into existence. You need to build toward it by having real systems deployed in the real world. Those real systems are not going to be level 5 at first or even level 3. This is where we are today, for general driving we only have level 2 and some level 3 if that.

Training is needed not only for the systems but also for drivers (including other drivers in traditional vehicles, they are also being trained unwittingly) and pedestrians, and regulators. Everyone is learning here. It’s not just an overnight, snap of the fingers kind of thing.

I believe the article speaks to the lack of data: we don’t know if L3 is safer. Safer than a drunk driver or a tired long haul, maybe. But a typical driver that would also use L3 responsibly, it may not be safer.

One other huge disadvantage of L3. As driver physically drive less and less while models learn, their skills will wither and when the car gives up control it will be to out of practice drivers.

This training in the wild sounds scary at first, until i remembered that we've done something similar with air travel, basically "training" with live people and fatalities. But over the years we did reduce those from 40 crashes/mil to 0.1 crashes per million flights. I would not want to fly on those early jets in the 50's, but enough people actually took that chance!

40 crashes per million flights has still got to be better than current road safety, at a very rough calculation the US for example had 128 road deaths per million people, which I'm admittedly not sure how to convert to make a fair comparison.

> However, there may be no other way to someday reach level 5 for general driving, other than to live with that middle zone for a while (keeping in mind that it’s already safer than humans driving alone) in order to conduct the training needed to reach level 5.

Gotta call bullshit on this. If human-piloted cars were bedecked in sensors, recording drivers' inputs and outputs, we could collect all the training we could need for L5 driving. It would take a ton of stringent curation, but that's going to be necessary anyway. It's not like Tesla has magically learned not to hit emergency vehicles stopped in traffic yet.

Or, alternatively, we can slowly make roads safe for any AI driven cars. Use appropriate marking on the roads everywhere, clearly visible signs, clearly visible crosswalks, physically separated lanes for bicycles and vehicle similar to bicycles etc. etc. etc.

Funny thing that we will simultaneously make roads safer for all cars and pedestrians and cyclists too.

It doesn't snow where you live, does it? And your proposal doesn't remotely address the "emergency vehicles stopped in traffic" issue.

Yesterday I was on a two lane country road where I saw a man standing on the opposite side of the road holding up a bright orange baseball cap in his hand and his other hand in a "stop" motion. I slowed down enough to where I didn't hit the crashed car which was just around the bend, but I may have hit it without the advance notice because the turn had very poor visibility. If I were an AI driver, I probably would have considered the man "not-an-obstacle" and promptly ignored him and hit the other car. After all, I'm trained to slow down when I see people in hi-vis with proper "slow traffic" signs. Some random person whose neighbor crashed and is waving the brightest color thing they have is not in the training data.

Or in other words, there are signals humans can send to other humans which can be understood even if you haven't seen that particular signal before, and those will be the biggest challenge for an AI driver, at least if it resembles the current generation of AI.

Fleet gathering of edge case data is a thing, and it helps with many cases. But of course you’re right some general intelligence is needed.

The greatest trick Musk played was marketing the lane-keeping assist feature of Tesla as “Autopilot” or FSD. It is the best lane-keeping assist I’ll give it that but FSD not in five years and not until commodity LIDAR and map-sensing. Mobileye deserves credit for jumpstarting Tesla’s lane-keeping assist early on.

> we are currently in the most dangerous phase

I did read the article. It’s ironic given her anti-Tesla stance how Tesla exactly follows what she recommends… “No hands-free L2” is pretty much exactly what Tesla urges and enforces its drivers to follow. Is the enforcement perfect? No, of course not; humans are simultaneously stupid and creative.

Try reading it again. She thinks that either the driver should control the car or the car should control the car. Hands on a wheel while the car steers is not a driver controlling the car. Tesla does exactly what she objects to.

Wayne did a study several years ago IIRC and determined that even their trained, professional drivers being paid for this exact task could not reliably, effectively and quickly take over driving when a self-driving car made a mistake. It’s a task that humans are really quite bad at.

I know I am. Steering a car on an average road: not a big deal. Taking over on time when a new, borrowed Tesla tries to mess up on that same road: not so easy. Also, it takes more concentration than just steering the car normally.

Hah, you try reading it again.

"So, no L3 [full self-driving, but the driver must be ready to take the wheel], which is too confusing, and no hands-free L2 [partial self-driving]."

Let's zoom in on the part so you can read it, too:

    "no hands-free"

Tesla requires its users to keep their hands on the wheel at all times, so they agree.

You have your opinion about this, which you explain, but it differs from her opinion as expressed in the article.

Key point (read the quote):

She puts hands-on L2 inside the bounds of what she accepts.

That is also exactly what Tesla currently provides.

Apparently data and people who use it to make decisions have an anti-Tesla stance.

meh. Some people. Selective using data does not make you immune from all errors and bias.

>FSD has been 5 years away for… 15 years now?

FSD is going to be enabled on HW2 anyday now. Oh look at that HW4 just rolled out.

> FSD has been 5 years away for… 15 years now?

the past few years it has been "later this year"

> FSD has been 5 years away for… 15 years now?

this was only Elon-time hallucinations that made for great marketing

you won't get to Mars on Elon-time either, but that doesn't mean you won't get there

>you won't get to Mars on Elon-time either, but that doesn't mean you won't get there

Doesn't mean you will either.

It wasn't just Musk, it was Lyft and Uber, Waymo and the rest are all still doing it. Lyft's CEO just put out a statement saying "Soon now".

It seems very ignorant to bet against AI given all the progress that have been made in that last year (!).

It seems overly optimistic to expect a breakthrough.

It seems intellectually lazy to assume that the present continues reliably into the future. You need a more concrete argument than "we've been making progress so far so we'll keep making progress", or if you're on the other side of the argument, you need something better than "this hasn't happened yet so it will never happen".

If the progress of neutral networks-based AI has thought me anything then I expect more and more applications where it's amazingly capable some time, capable most of the time and completely fails still relatively often.

I am really skeptical at this point if it can ever master something to the point where we can safely trust our lives with it.

Two entirely separate fields, both with a similar problem: not good enough. Impressive, but makes mistakes and an 80% solution is not what you want for self-driving cars.

I would struggle to risk my life and my family on even a 99% solution. Is a FSD wants to drive me head on to a railway one day, which has happened, I think I'd rather have been at the wheel. That's just me though. I view my life as mission critical.

Most people in 1969 would probably say the same about betting against humans on Mars well before the end of the 20th century.

The progress recently has been with LLM. Great when you want to talk to your car or car needs to read street sign. Not useful if want to drive the car. It is possible that the techniques transfer over but self-driving is pretty different domain. My guess is that can share general techniques but the LLM breakthrough isn’t useful for self-driving.

Reply on news.ycombinator.com