Right, so assuming LIDAR caught her: I'd imagine the algorithm presumed that she wouldn't cross the center line till she did cross the line, I don't know what to think the algorithm would do there after?
I don't have the link handy, but I was reading a webpage yesterday (related but not about this crash) which showed Google's self driving car's "view" of a road scene - it's clearly painted different color boxes and identified pedestrians, bicycles, other cars - along with "fences" where it had determined it'd need to slow or stop based on all those objects.
Either Uber's gear is _way_ less sophisticated (to the point of being too dangerous to use in public), it was faulty (but being used anyway, either because its self test is also faulty, or because the driuver/company ignored fault warnings) - or _perhaps_ Google's marketing material is faked and _everybodies_ self driving tech is inadequate?
I think this is a very good possibility considering that autonomous vehicles is the goal of the company and they're racing to get to that point before they run out of investment money. They have a lot of incentive to take short cuts or outright lie about their progress.
(2016)
1. System was off
2. Point clouds were not being registered correctly (at all!)
3. It was actually in manual mode -- safety driver didn't realize or didnt react fast enough.
4. Planning module failed
4. Worst outcome in my opinion: Point cloud registered correctly, obstacle map generated correctly, system was on, planner spit out a path but the path took them through the bicycle.
A website that "does something weird" when you use a single quote in your password... That _could_ be "the only situation you have to worry about". It is _way_ more often a sign of at least the whole category of SQLi bugs, and likely indicative that the devs are not aware of _any_ of the other categories of errors from the OWASP top 10 lists, and you should soon expect to find XSS, CSRF, insecure deserialisation, and pretty much every other common web security error.
If you had to bet on it - would you bet this incident is more likely to be indicative of a "person pushing a bicycle in the dark" bug, or that there's a whole category of "person with an object is not reliably recognised as a person" or "two recognised objects (bicycle and person) not in an expected place or moving in an expected fashion for either of them - gets ignored" bug?
And how much do you want to bet it's all being categorised by machine learning, so the people who built it cant even tell which kind of bug it is, or how it got it wrong, so they'll just add a few hundred bits of video of "people pushing bikes" data to the training set and a dozen or so of them to the testing set and say "we've fixed it!"
If that's your idea of adequate, you'd be safer just vowing to get drunk every time you drive from now on, since a modest BAC increases accident rates, but not by a factor of FIFTY!
https://www.wired.com/story/tesla-autopilot-why-crash-radar/
https://www.theguardian.com/technology/2016/jun/30/tesla-aut...
This is going to sound bad, but I hope this is just Uber’s usual criminal incompetence and dishonesty, and not a broader problem with the technology. Of the possible outcomes, that would be the least awful. If it’s just Uber moving fast and killing someone, they’re done (no loss there), but the underlying technology has a future in our lifetimes. If not...
I, for one, certainly won't be betting against you there...
From the reports of cars running red lights and then this I would imagine they have an extremely high level of "risk" (what it takes for the car to take actions in order to avoid something/stop) that is acceptable.
What would be far worse than a hardware or sensor failure would be to learn that Uber is instead teaching its cabs to fly through the streets with abandon. Instead of having cars that drive like a nice, thoughtful citizen we'll have a bunch of vehicles zooming through the streets like a pissed of cabby in Russia.