Hacked billboards make Teslas see phantom objects causing them to swerve or stop
newsweek.com
newsweek.com
I can see this happening as an opening shot to an action movie.
Some reporter finds out that $government is doing $bad_thing. They pack up the evidence, rush out the door to talk to their editor. A slick, self-driving car pulls up as they exit the building, they enter and pull away.
It is a rainy night, and the car zips along the city roads along thousands of others; just another regular evening. Cars passing each other at break-neck speeds with safety margins that are only really safe with positronic brains at the wheel.
We see the street, as seen by the computer. Bounding boxes and labels hovering over multiple viewpoint video feeds, all drawn in slick black & blue cyber aesthetic. Suddenly something changes at the left of the image, alerts go off.
Cut to the exterior, super slow motion shot of the care from back right angle, with a big billboard visible on the left side of the road. It shows cryptic patterns that seem to depict running children, but never the whole child, just portions of them, in changing shapes and positions, with neural network adversarial noise covering the rest of the image.
Cut back to the board computer's view, still slow motion. Red bounding boxes surrounding the billboard, and then a big red warning flashes in the center of the screen:
COLLISION IMMINENT.
All kinds of meters at the bottom of the dashboard start going wild as the car tries to find a safe exit state, and with the slow-motion slowly approaching realtime again. It swerves to the right. The image gets rocked as it gets hit by another passing car. Another hit. The world on screen turns as the car goes flying.
Static noise fills the screen.
Next scene:
A news broadcast announces that there was a high casualty in accident on Suchandsuch Road last night. Investigators find that it was most probably caused by a repair shop installing a faulty aftermarket AI module into a car.
Original research: https://www.nassiben.com/phantoms
I almost had an accident when a billboard that was dark suddenly displayed a very bright image on my side. My body reacted moving suddenly the steering wheel as a reflex action.
At another one having some big image being moved actually made my eyes follow the movement instead of the road with cars on it. I had to make work in order to focus on the road and not the screen.
It also makes your eyes to loose your vision in the dark.
- terrifying: finding services in far flung places could be nearly impossible
- likely to recreate both of the problems I’d want it to address in potentially more harmful ways: for road safety, it means discovering services would depend even more on electronic devices inside the vehicle and further from the safe driving field of vision; for profit opportunism preying on drivers’ eyes, it consolidates the market much further than it already is, and with even greater harmful incentive
[1] https://textslashplain.com/2017/01/14/the-line-of-death/
One would hope Tesla's system has some depth awareness. Honestly I'd be impressed how they could do so much if it didn't.
[0] https://www.wired.com/story/tesla-model-x-autopilot-phantom-...
The only difference here is that a mere 500 ms flash elicites a response, where a human wouldn't notice. on balance, a 500 ms response time does seem like a feature more than a bug compared to a human.
Are self-driving computers superior to humans and make less mistakes, or is it OK for a computer to be tricked by something some humans would also not detect?
Btw. I think I can tell if there is a real stop-sign or something on a billboard after some 'huh'-moment and without slamming on the brakes. And humans still go circles around AI regarding plausibility-checking very out-of-the-ordinary events.
Self-driving cars aren't superior.
There is 1 death in 100 million miles driven on roads; self-driving cars are currently operating around 1 death per million.
Personally, I do not expect self-driving cars this century to get anywhere near human safety levels (assuming cities/etc. remain as-is).
and
https://www.vox.com/recode/2019/5/17/18564501/self-driving-c...
I haven't read it all -- my numbers come from another paper, I can't recall. But it's talking in the right ballparks.
The idea is that if you can take an image and predict the distances to the objects in it, you can then create an image that is a transpose of the original based on your predictions, but at a different distance. In case of Tesla moving forward, that would be a split second later, closer to the original image. As the car moves forward, you actually take that second image and compare the virtual to the original.
At this point you probably get it wrong, but you adjust your predictions and recalculate until your virtual transposed image looks exactly like the real image, at which point you've got the distances pretty darn close.
I remember watching a talk about their computer vision, and it was pretty impressive. The challenge is that despite knowing where the billboards are, they still look that way because sometimes, in obscure situations, in little old towns with fuzzy rules, road signs actually do appear in weird places, and that's where the car often trips up because without sufficient data it just can't build a prediction model to know when to respond, and when to suppress the new input.
I have a side project where I manage to get a helluva lot of depth info using a single photo from a single camera (monocular photogrammetry).
But it is the unexpected we need to train computer vision for to compete with humans. We can have rules for things like billboards, sure, but those will never be completely implemented everywhere.
The software just needs to be improved to identify something as billboard.
Sometimes they're are used for good, like making drivers slow down: https://www.insider.com/optical-illusions-3d-crosswalk-drivi...
But other times they cause crashes: https://imgur.com/a/kYr94
there are lot of similar failure modes in individual humans. The difference is individual driver errors are not correlated, a bad reflection tricks a few humans at once instead of tens of thousands of cars running the same model. There is significantly more brittleness in a fleet of cars than in a population of humans, because the human population has diversity in judgement and experience.
This is not simple to fix because this uniformity is actually a feature of automated systems, it makes them explainable, produces expected results and conform and cheap, training a human for 20 years is more expensive than training all the cars once. However it also makes them collectively vulnerable, which is why heavy machines tend to be locked away on factory floors.
it seems to be the opposite of assistive. How can a person know all the weird things that will make a computer vision system badly fail. It shouldn't be used at all if that is the case
The effect is more like: What is this for strange art!?
"They're also so fleeting that they're difficult for the human eye to detect, and leave behind very little evidence."
We've already got people who do things purely for the lulz (swatting, anybody) who could easily justify away these kinds of activities.
I've had some terrifying software problems in Audis, including random shutdowns while driving due to faulty sensors, how often do you think someone who 'lost control of their car', which implies that there was human error, died because their car's electronics or software were faulty?
That number definitely is not zero.
My old Celica tried to murder me on the M62 and Toyota denied the problem for years (the gas pedals got stuck under the floor mats). And that was just a really obvious (in retrospect) design flaw - for how long could a car maker viably deny safety problems in their software?
There's an argument to be made that it should be covered by cyber-attack laws. You aren't allowed to deliberately take control of someone else's computer without their permission.
Consider a hypothetical example: an author without use of their hands, who uses text-to-speech to write books and to control their computer. Imagine if they put you on speakerphone and you read out a sequence of commands to delete their work. That certainly wouldn't be covered under freedom of expression.
There's an obvious difference in degree, as spamming someone by hijacking their smart speaker isn't all that costly, but it's similar in category.
[0] https://en.wikipedia.org/wiki/Burger_King_advertising#2000s%...
Yes, but smart speakers should not rely on laws to prevent successfull attacks.
They are not, any more running someone through a printing press to kill them is “free press”. Regulating intentional harm without reference to the particular means is not a violation of free speech or other expressive rights just because someone uses a tool that could instead be used as an expressive medium as a mechanism of inflicting the harm.
This sounds like a horror movie scene, and the killer would make some terrible pun at the end.
"Extra extra, read all about it...You're dead."
This seems like hanging a rock painted with political slogans off the side of a bridge (presumably by some sort of rope) and letting people drive into it.
What is true or false isn't always clear though, even if there's a popular or official consensus.
The point of having a public forum is to give new ideas a chance, as unheard of or unpopular as they may be. Of course, with that you get the risk of misinformation as well.
As a society we must choose how much risk we're willing to tolerate, and really consider if the the downsides of the level of free speech we have today are really so much different than what we've historically dealt with.
While the veracity of a statement is important, the problem is not really about "misinformation". Technologists and intellectuals are used to viewing speech as an form of information transfer. However, speech can also be performative[1]; saying "not-guilty" to a judge or "I do" during your wedding are not really about conveying information - by saying those statements you are performing an action.
Falsely shouting "fire" is a false statement, but the problem happens when that statement is also functioning as the act of reckless endangerment (or incitement to riot, or whatever). Nobody cares if someone e.g. writes an op-ed that falsely claims the theater is on fire. It's the same misinformation, but the op-ed isn't also functioning as a performative act.
Far too often in these discussions, the different form of speech are conflated. A lot of the "balancing freedom vs safety" problem is a false dilemma that becomes a lot easier to reason when speech-as-information is separated from speech-as-action[2].
[1] https://en.wikipedia.org/wiki/Performativity
[2] Factoring out phatic language is also important ( https://scienceblogs.com/clock/2007/05/31/more-than-just-res... )
So since the ideas behind scientific racism, the dark enlightenment, white supremacy, Nazism and anti-semitism, and any prejudice based on conservative or religious ideals (such as sexual or gender discrimination or anti LGBT prejudice) are neither new nor unheard of, it's OK to stop giving them a chance?
I mean... flat earth? Do we really need to make sure the flat earthers have as wide a platform as they want just in case they turn out to have been right all along? Do we really need to vigorously debate every new iteration of "maybe the Jews do deserve to be driven into the sea" or "maybe eugenics is a good idea" that crawls out from under the rocks of society every few years? Do we need to keep the flame of "Bill Gates created COVID as a pretext to tag everyone with mind-control chips to make it easier to harvest adrenochrome for satanic blood magick" alive for future generations to ponder the possible wisdom of?
Maybe we can agree that, while there may be subtle cases where legitimate scientific or political ideas which run afoul of the status quo and public acceptance need protection in a free market of ideas, most of what's actually being defended in these discussions is regressive garbage that can be tossed back into the wastebin of history from whence it came, with nothing of value lost?
If we were to take your argument back in time and apply it, what would we see?
"I mean...heliocentricism, really? Do we have to give Copernicus slosh any thought, we all know that God created the earth and then the heavens so it is only natural that they orbit the earth."
On the front page at the same time is a story about psilocybin and "What if a Pill Can Change Your Politics or Religious Beliefs?".
Right now we accept that sexual orientation is not a choice. What if in the future we find that exposure to chemical/compounds can cause people to become more gender norm conforming? What about less conforming to historical norms?
How would you propose we handle those possible futures? A drug that turns you gay is straight out of Alex Jones. A drug that turns men straight is straight out of conversion therapy.
OK, so I see you've decided to ignore the context of my comment, pretend I was referring to some absolute "TRUTH" then strawman it.
Yet everything you've listed, being scientific and therefore part of a framework in which those ideas could be proven or disproven, are part of what I mentioned at the end of my article, which I will quote verbatim for you: cases where legitimate scientific or political ideas which run afoul of the status quo and public acceptance need protection in a free market of ideas and thus explicitly not germane to my point.
How do I suppose we handle science? Science. But then nothing I listed was actually science, nor is there any valid controversy around any of them, as science has invalidated practically all of it already. There is no universe in which it turns out Hitler was right all along if only we'd listened.
So let me flip the script on you - how long do you believe we should let ideas like white supremacy, anti-semitism, lunatic conspiracy theories like QAnon, anti-vaxx etc remain in the marketplace?
Or even keeping it within the bounds of science - do you believe we should still be arguing the merits of the luminiferous aether and miasma theory, or "teaching the controversy" around creationism?
It might be an example for speech that ought not be protected, but it's not an example of unprotected speech grounded in either the preexisting case law or any precedential rule in Schenk that is still operative, so it's not a useful touchpoint for analysis. It's pure unsupported dicta sandwiched between two well-referenced points of law in the decision.
> Speech that is false and intentionally causes a panic should not be protected.
Intent is not an element of the well-known hypothetical described in Schenk, nor the even more abbreviated form derived from it raised in this thread.
70% of facebook posts and newspapers instantly banned.
Corporations should never pretend to be the guardians of truth. Bias will just spew from there about whats supposedly real and whats supposedly fake.
A terrorist on the other hand would probably be dissatisfied with the not-quite-catastophic outcome. If you want to cause real havoc, there are far more valuable targets to hack into.
I'm not saying that this could never possibly happen, but it's not much of a real threat exactly because effort vs reward is unfavorable.
I've no idea whether that'd be a highway crime, a computer misuse crime, or just attempted murder - that'd be a new one for lawyers in every country to argue.
Admittedly its only shown for half a second but I imagine a distracted driver who looked up suddenly could be equally fooled.
Having video ads on screens that drivers can see and be distracted by is the real problem.
This is a solvable problem. Just another example of where "AI" is lacking a sense of intuition.
I should start carrying a stop sign in my pack, have some fun on the highway.
I would say that this billboard hack means anyone could install a banner on their balcony to get a tesla to swerve onto a neighbour’s house.
However, I believe this will be fixed in a later update.
Maybe. I hope so. But from my understanding, context is the hardest part of computer vision.
You either detect a full-stopping-distance's worth of clear road through reliable (non visual) ranging with dense enough samples to preclude wheel-catching voids ... or you will continue to run into "objects" which slipped through your classifier.
I can accept reduction in this margin (e.g. in dense traffic flow) when it is first shown to work with margin.
I'm interested to hear how you decide what's drivable without detecting things that decide if roads are undrivable.
I'm sure what I said was poorly worded. What I meant was that reliable ranging with dense sampling should be the primary mechanism that self-driving vehicles use to determine what areas of the road are safe to drive on, and probably what is even a road at all. I think efforts to avoid this are based on cost, not safety or functionality.
Other signals are just too transient to be safe. Vision-based classification-first regimes are not guaranteed to detect hazards as reliably as a point ranging device, is that controversial?
Potholes exist, but are geometric negative space. Clues to their existence can be provided by imagery or other sensors (which are subject to all the weather effects also, by the way), but a positive detection of good road or pothole should have priority.
Semantically, it's a meaningless distinction. But in implementation it's much easier to detect the shape of the upcoming terrain using, say, lidar or regular old multi-camera disparity. Flat and following your map: that's driveable. I'm guessing many take this approach, but it's seeming more likely that Tesla does not, and instead trusts monocular or arrayed cameras to track objects though frames from detection-first regimes. Seems dangerous.
I'm glad you were interested.
Can you guess why blindly driving down each of these will be dangerous?
You can actually correct those instances by using reliable ranging devices with dense enough sampling to ensure flat surfaces without intermediate obstacles.
To say that there are no other factors at play is foolish, of course, but it's also foolish to say that somehow we shouldn't be doing this.
Walk before you run is the essence of the argument here.
Realising that a flat surface is drivable is probably 1% of 1% of what an autonomous car does on a road. Same goes for humans, I'd say.
Walking on the highway because you got spooked is the essence of the reason why you're dead.
I can rephrase: Reliable ranging should override other sensor mechanisms for detecting driveable space and presence of hazards.
Tracking with dense ranging (e.g., LIDAR) is not as challenging as the alternative: ensuring that non-ranging system will reliably segment all possible hazards.
Any money spent on infrastructure for self-driving cars would be better spent on public transportation.
It's very hard to really gauge the impact this has on human productivity, especially given that in public transit you don't need to control your vehicle and so can be doing something else (although from my anecdotal experience, the vast majority of public transiters are doing nothing productive or interesting on their commutes).
But then there is also that, when a global pandemic happens and localities lockdown public transit, people with cars can still go places (like a National Park).
Point being, the problem is absolutely not one-dimensional and so just comparing aggregate expense is not very useful at all.
I think the goal for transport should be to maximize human productivity and mobility, and given the above and other realities, that almost always means a strong hybrid approach, not one where you relegate cars to a lesser status.
I suspect that to make real progress we will need to move away from our current assumptions. I'm not from the US, where perhaps the expectations and practicalities of transportation can be quite different to somewhere like Europe. But even here, there is a tendency to conflate mass transportation (buses, trains, etc.) with public transportation (vehicles run as a service by someone else, rather than owned personally and reserved exclusively for the use of their owner). As the existence of alternatives like taxis and school minibuses demonstrates, these are really two separate axes. And that's before we get to all the complexities of how we pay for transportation, both individually and as a society collectively.
There are some big advantages to providing individual transportation tailored to each specific journey. Going door to door at an exact time of their choosing can obviously be more efficient, sometimes several times more efficient, for the individual traveller than having to connect to and travel via a network of predefined locations on a predetermined timetable. There are also some health and security implications for travelling individually, and of course it can be much more comfortable because you have plenty of space, a guaranteed seat, your preferred temperature and ventilation levels, etc.
There are also some big downsides to individual transportation using the options we typically have available today. In particular, the individual vehicles we use right now are often not well suited to any particular journey. People are more likely to own a single vehicle, which they use for anything from taking their whole family on a long distance journey to a one-person drive to work or for shopping. That can be very inefficient in terms of space usage, environmental impact and operating costs. But then owning multiple vehicles, suitably sized and featured for different types of journey, is also very inefficient if each of them then spends most of the time sitting idle in a garage somewhere. And there are safety implications for using a smaller personal vehicle on the same roads as bigger, more dangerous vehicles operated by fallible drivers, as anyone who cycles around a big city can testify.
On top of that, you have an emergent system where there are millions of individual journeys happening on the same infrastructure with at best some very local co-ordination via things like lights at junctions. If you look at the mathematics underlying transport networks, you can see how this can lead to all kinds of surprising and sometimes very unhelpful overall outcomes, even if everyone is acting logically as an individual part of the system.
But there is no rule that says things always have to be this way. Sometimes mass transit uses dedicated infrastructure, as with trains and planes, and sometimes it's shared with individual transport, as with most buses and trams. Future infrastructure could be designed to support, as one possible example, both small and efficient personal vehicles and larger vehicles with space for more passengers and/or cargo that could be summoned on request, with preferred routes and timing specified by each traveller, but with the actual co-ordination of the vehicles administered centrally and the vehicles themselves operating autonomously.
It's obviously a huge jump from where we are today to such a system, and there would be all kinds of difficult questions about how we could migrate from one to the other even if we somehow all agreed on where we were ultimately trying to reach. But one thing is for sure: we can't even start working towards a system like that as long as we stick to preconceived notions of public transportation as mass transit with fixed timetables on large vehicles like buses or trains, and private transport as the family car as we know it today.
Isn't there a bureaucratic problem hiding in that observation? Some money is saved by police and by individuals who would have crashed, but the departments responsible for upgrading the infrastructure will never get a return on their investment. In practice that'll mean higher taxes to support some new-fangled tech that doesn't immediately help the majority of people, and I'd be surprised if that gets support.
The roads could have captors/sensors/tags added gradually to them. For example, some near field devices planted in the road and broadcasting information such as the name of the road, mile mark, line number/side or traffic lights broadcasting if they are at red or yellow or green or signs such as Stops broadcasting their presence.
This would help autonomous vehicles a lot (instead of having to scan the infrastructure and deduce the environment, the infrastructure will broadcast the environment, at least the static parts, directly), while at the same time not be that costly (compared to the initial cost of the road or even its maintenance costs).
Our bus system where I live travels with many busses nearly empty. This is what happens when they are funded by a percentage of property taxes rather than individuals paying per ride. A perfect example of government waste.
I think the issue is that a dedicated test area will just never reflect the full set of problems you encounter in the real world, and for things like LED billboards it's going to be a lot cheaper to test in the real world than to buy and install your own. On the flip side, there's next to no money in developing "vehicle-dedicated infrastructure," so no one's that interested in testing on it exclusively.
~37-55 traffic lights, ~9km, 108.11p, city driving on 1-7 lanes (mostly 2-4) with on and off street bicycle lanes (Please don't hit me test cars!).
As a resident on the test track I have only once seen it in use. It would be fun to acquire a suitable radio and try listening in to what's being broadcast but that's a project for another day.
(Only 50% joking.)
The rise of AV testbed cities
https://www.axios.com/autonomous-vehicles-japan-china-testbe...
If it were a stop sign in a billboard ad, humans would ignore it, though.
This is currently a research frontier for AI so us non-experts likely won't be able to say a ton about it.
I thought this was a good talk on the issue: https://www.youtube.com/watch?v=SS9DMr4VkbY
Autonomous vehicles have the potential to reduce deaths and injuries dramatically and increasing the time to get there will cause thousands to millions of more deaths and an order of magnitude more injuries, not to mention property damage.
(I have no idea where lidar for self-driving cars falls within that balance, although I can't help but notice that human drivers do not utilize lidar.)
I suspect that right now, ML based approaches to depth perception would not pass the test to standards we would like, in practice requiring lidar. But, this would also provide an option to switch to an ML based gets good enough.
From a driver responsibility point of view, it is a fancy cruise control, there is no question about autonomous certification because the car isn't autonomous.
If one day, Tesla thinks it is OK for drivers not to pay attention and wants to make it legal, that's when the question about what kind of tech is reasonable will be asked. But for now, it looks like we are far from it. There is a huge gap between "works well most of the time" and full autonomy.
Whether this is false advertising or even just moral is up in the air, but Tesla is making no attempts at educating consumers about the limits of their autonomy other than the bare minimum (yeah you technically have to keep both hands on the wheel, but that mechanism is easily defeated and you can even buy clips on Amazon that do it for you). I'm pretty sure Elon even liked a tweet about a Pornhub video where someone banged a model on the highway with autopilot. Which is an excellent day for PR and a disastrous day for legal.
The point being, sure, academically they are a far way away, but they are trying to convince consumers Level 5 is just around the corner.
If a human has to sit there and monitor the car without interacting with it, then we shouldn't allow it on the roads. That means skipping level 2 and probably level 3. Go the Waymo route and go straight to full self driving cars.
Great for detecting if an object is in your way - but it cannot read a stop sign, you still need a camera system for that - so they still have to harden it against this hack (and who knows how many to come).
True, but LIDAR should make that detection more robust.
For example, if a stop sign is on the same plane as the object that surrounds it, that's a strong signal that it is not a real stop sign but rather a projected or displayed one.
That's much harder to detect with computer vision alone, especially if they're dealing with hackers who can test it with a real car and continually iterate on it to find different ways to trick it.
It's conceivable that in denser or older areas, signs are sometimes placed on structures because there isn't room for a pole.
Apple Maps shows stop signs and traffic lights on the map. A database like that would be another strong signal, but also not always accurate.
It makes me think that the first real wide scale self driving vehicle deployment will have to come from a nation state rather than a corporation; the easiest way to solve all of these problems is by putting heavy investments into smart road infrastructure such that the car depends on sensors embedded in the signs or the asphalt rather than any camera feed. A school bus could just have a transmitter to indicate whether its lights are on or not; construction workers could be given road signs that transmit similar signals or better yet use a portal to construct a virtual geofence around wherever the construction is happening before it begins. These are all easily solvable problems in today's world; what is missing is the political capital to invest in these solutions despite the insane ROI.
This infrastructure will need distributed two-phase commit. For safety-critical issues like school buses, eventual consensus won't be good enough. And then you'll need an infrastructure that detects when an older or simply defective car is present but not participating in the protocol. These problems are all solvable, but they're by no means trivial.
That is exactly why they can't interact with real world cases of fully valid plastic signs coming out of nowhere.
This isn't a perception issue. It's an insufficient model for the world, and the software is obviously biased towards obeying critical traffic controls at lower confidence levels. It would be interesting to see what would happen if a (higher) speed limit sign was flashed in the same way.
Still I wonder if it has failure modes that humans lack, which could result in situations where it does inexplicable things.
Almost certainly :)
Maybe .... before acting on a Stop sign sensed by the cameras the control system would confirm, via LIDAR, there is a post with a plate mounted on it in a position which corresponds to where the camera thought it saw a Stop sign ?
Can LIDAR do that level of sensing, ie unambiguously find a metal post/plate out of all the other stuff which might be in front of the car ?
And assuming my guess about how LIDAR would help is correct what happens when an obstruction between the car and the Stop sign means that LIDAR can't 'see' the post but the cameras can see the sign ?
Well surely that's what we should be testing. Is a self-driving car system safer than a human, or not? You're correct that self-driving car systems lack a "general learning and intelligence model," but they also have several obvious advantages over humans, like potentially better-positioned cameras, a more reliable "attention" system, faster reaction times, potentially more "experience" with different road conditions and scenarios, etc.
Humans do get very clunky and slow when that depth perception is taken away. One example is night vision goggles which basically just project a flat screen onto your eyeballs. Many people report that they become susceptible to tripping, falling, hitting objects, or just straight up walking into trees with NVGs, in part because the loss of depth perception takes such a toll on their brain that they can't adapt in time. It takes hours if not days to fully adapt to NVGs, and even then you can't operate as quickly as you can in daylight unless you practice consistently (think SWAT or Navy Seals type training).
Cameras lack any perception; perception is a function of brains.
But a single camera with appropriate processing can provide a wide array of depth cues (stereoscopy adds more.)
> Many people report that they become susceptible to tripping, falling, hitting objects, or just straight up walking into trees with NVGs, in part because the loss of depth perception takes such a toll on their brain that they can't adapt in time.
While loss of depth cues plays some role here, a much bigger factor, AIUI, is the narrow field of view. “Panoramic” NVGs have around 100° horizontal and 40° vertical field of view (non-panoramic NVGs often have around a 40° horizontal FoV), people have around (with eye but not head movement) a 210° horizontal and 120° vertical field of view. Looking directly ahead with even panoramic NVGs as you normally would without them, you literally cannot see lots of things, especially tripping hazards, and objects to your sides that you might laterally move into to avoid the things you can see in front of you.
The reason people become clumsy when using night vision devices is because they have a tiny field of view that only moves when your head moves. You can close one eye and walk around just fine. But if you attach two toilet paper tubes to your face, you'll have trouble balancing and avoiding obstacles. The clumsiness is caused by the lack of peripheral vision and the inability to quickly glance (since the only way to change what you see is to turn your head).
Furthermore, everything about the road is designed to help us humans. The size, color, placement, etc... of road signs and markings is specially designed for human brains to process.
As for the hardware, our eyes have very good resolution, field of view and an unmatched dynamic range. Far better than the stuff in Teslas.
So I'm not saying that it is impossible for computer vision to get away with just using cameras, but it is competing with what our brain does best. So a little help in a form of a LIDAR isn't too much IMHO.
You can of course pick out discrete examples of humans failing tasks like these, but entire classes of computers routinely fail at things like this.
Not that long 'til somebody points a projector at the ground in front of a Tesla and it crosses into oncoming traffic. You could even mount that projector on a panel van, and use image recognition to target Teslas and otherwise stay inactive
I don't understand why people come up with all these ridiculous "what if" ideas against autonomous cars as if it's anything new.
It's already exceedingly easy to foul up a human driver. Stand next to a freeway and hold up a stop sign. Drop a foam mannequin from a bridge over the road. Paint some bogus lane lines. Shine a laser at their face.
Any of the above will cause accidents with humans, and you'll be arrested and charged with a felony. Deliberate attempts to compromise autonomous car systems carrying humans inside should be treated no differently.
The panel van (or drone) attack is highly targeted, and doesn't stick around like paint does.
These aren't just "ridiculous what ifs," they're a rather plausible method for freshly unemployed Luddites.
Sure, leading a self-driving car astray is more of an accomplishment than blinding a human, but comparing crimes by how entertaining they are to a bored unemployed DIY sociopath does not seem very reasonable to me.
Sure, your chances of getting into an accident while driving your own car are higher, and some dummy could throw a turkey onto your windshield and kill you ...etc, but humans don't think in probabilities (well, outside of Hacker News humans), and while driving their on car humans have an illusion of control, when the car is driving itself and you are sleeping, that control is gone.
Have you ever flown in a plane before? Some people are more nervous than others about giving up control but billions of people still do it because the benefits vastly outweigh slight anxiousness about that billion to one chance. People will not want to drive in an autonomous vehicle right up until they see they can give up their $40k car and hail a driverless taxi to go anywhere for pennies on the dollar.
With planes I think most people realize that the trained pilot is a better pilot than they are, so they feel more at ease giving up control to someone that is better than them. People just need to see that similarly, software drives cars better than humans.
Seem simple to handle, as the car computer see an image, it checks if the image show up in a second scan. Since the computer scans are very fast they should not effect to computer processing/reacting to driving the car.
There will be a lot of money thrown on adversarial attack mitigation. But how much is too much? At some point it might become so expensive, that it will be cheaper to switch to guiderail-based self driving, which can be overseen and secured centrally. And self-driving planes are probably going to be a nonstarter
My general feeling is that these sort of intrusive, dangerous, anti-ecological (such a billboard consumes as much power as a family of 4), anti-social (one less worker to change billboards) ads should be banned, period.
The question becomes, how do we give an AI the concept of a BS detector? How can we train it to know, somehow, that the classification it has just made with a high degree of confidence makes no sense? This seems like a very difficult problem to solve.
Less glibly, yes. There are particular arrangements of roads which are known to get people killed by cars. For example, an intersection at a particular angle with particular speed limits cause cars to stay hidden in each others' a-pillars as they approach the intersection. There are bits of highway that are known to cause human drivers to drift; this is why many stretches of highway now have textured strips down the side to notify drivers that they've moved out of their lane.
Don't want to drive, take the bus.
The idea that we all need our personal chariots (and now robotic chauffeurs) seems to me to be the height of irresponsibility.
And, yes, I am guilty of owning a car myself and feel guilt about it. But for some reason the self-driving bit just seems to take it to another level for me. Maybe too there's an air of elitism about it since we are in fact talking about cars that cost what most people make in an entire year.
It's also about enabling Uber-style intelligent carpooling so that you can save energy while still maintaining similar times to driving, especially in places with poor or non-existent public transportation.
It's also about reducing the number of cars in the world. You probably use your car for less than 5% of your life. Most of the time it just sits there taking up space. Why do you need to own it? Imagine a world where you didn't need to own, maintain, or park. We can reclaim parking lots and plant trees, build housing, or other things. We can also reclaim a lot of our time spent in traffic jams or looking for parking and do something more useful with it.
It's also about enabling people who are unfit to drive to have a better quality of life. What if you've had a little to drink? What if you're down with a fever and a headache and can't concentrate on the road? What if you're sleepy? What if you are prone to seizures? What if you're blind, deaf, have poor motor skills? What if you're a visitor to an area without a public transportation system and not confident renting a car and driving?
To add, back in the old days when I still went into the heart of my local metropolis, Uber and Lyft style carpooling had the opposite effect of "reducing the number of cars". The streets are littered with pee bottles and packed with desperate drivers looking to score a quick buck, making u-turns across traffic while looking at their phones.
Because our primitive brains instinctively want to own things. We can share with family, and with good friends, but not with strangers.
Another thing: car is used not only to drive, but also as a storage space for things that you want to have always with you: a bottle of water, duffel bag with gym clothes, umbrella, etc. With rented car pools you'd have to carry all that with you.
Also, society where no one owns a car or an apartment, where everything is rented, "pooled", would be a totalitarian dystopia - try to misbehave, post a negative comment about government on Facebook and you're banned from a car sharing service as a punishment. Seizing your property is of course also possible, but much harder.
Personally, I think a lot of people would take that trade. I would.
Yes, there are people who like lugging everything but the kitchen sink with them around. For a lot of people in the world though "things you always want to have with you" fits in a handbag or backpack.
A big issue with public transport is the “first/last mile” problem. You want to take the light rail to commute 25 miles to work but the station is two miles away from your house and the closest station to your work is over a mile. “Just walk” does not work for everyone, either because of weather (getting soaked by rain or sweating too much in your work clothes), or physical abilities, or time. So are you looking to take the bus to the station and from the station to work? How much time does that add? Do the schedules even line up well? If you buy a bike, is it allowed on the light rail, is there a safe place at your work for it? Do you drive a car and leave it at the station, and what do you do on the other side to get to the office?
Autonomous vehicles can/should be used in conjunction with public transport to solve the first/last mile problem. Hail a small electric autonomous car to take you to the light rail station, and hop into another one at the other end that takes you directly to work. It is still convenient for the individual and it still eliminates 23 miles of vehicle driving in a dense urban environment and removes the need for a parking space in a location that space is likely a premium.
Same with busses. They take forever because they have to stop so often. They have to stop so often because people can’t walk long distances to a more central location for pickup. People in the lowest incomes that, out of necessity, have to use the bus system may commute for multiple hours simply because of these inefficiencies. But if you had small 10 person shuttles servicing a limited area to bring local people to and from their front door to a more central bus location, you could reduce the number of stops a bus makes over a commuter distance by an order of magnitude and turn a 2 hour commute into a 40 minute commute.
Self-driving “cars” include taxis (that's one of the prime motivating use cases), trucks (another big one), and busses (not as frequently talked about, but once the technology is there, it's going to get applied to them, as well.)
It's not just about individually-owned vehicles for people who want to own, but not drive, a personal vehicle.
It seems to be popular to want to automate all the "working class" jobs though.
Take cell phones for example, they were big, expensive, had only one function, and only available for the rich just about 30 years ago. Now they are cheap, powerful, pocket supercomputers available to mostly everyone.
Another example is farming and supply chain. 100 years ago you were pretty limited to the foods available in the local store, but now you can get anything your heart might desire, be that bananas from Mexico for 69 cents, or Sake from Japan for $20. The world has greatly benefitted from advancements in technology that was once only available to the richest.
Most of the mid-class luxuries we enjoy today, from airlines, to hot tubs, to game consoles and books on Kindle, all of that had to be developed and brought to light, first expensive, and then gradually cheaper and cheaper.
Eventually even the buses will drive themselves, and that will only happen if we invest sufficient energy and resources into making the tech available and improving today.
Well we are selfish creatures. It makes perfect sense, what is the purpose of anything if not for you?
I don't think we'll be satisfied by nothing less than everything, and why should we?
>Don't want to drive, take the bus.
And what if my area has shitty bus service?
But there is no evidence of this, or that they're even as safe as human drivers. In fact, Teslas are involved in more accidents than other luxury cars.
I strongly advise that anyone who pays attention to national politics diversify at least 50% of that energy into local issues.
I never rode in our vehicles... for a reason.
It is entirely possible, however, that the computers and software systems available today can't be better than humans.
The important metric is accidents per mile driven. Right now, Teslas on autopilot seem to be significantly safer than human drivers. From Tesla's vehicle safety report[1]:
> ...we registered one accident for every 4.53 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.27 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.56 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 479,000 miles.
Yes autopilot has issues and limitations, but it's already 3x safer than unaided drivers. Also, it seems to be improving in safety. Two years ago, autopilot's crash rate was only 1.5x lower than unaided Tesla drivers.
Weather, type of road and so on has huge effect on number of accidents. Unless those are taken in consideration it's only lies...
A portion of this is a result of the ability for young adults to purchases $30-40,000 cars: 16-19 yo drivers make up around 2/5 of the total accidents in the US.
https://aaafoundation.org/rates-motor-vehicle-crashes-injuri...
Additionally, 4 door sedans in general (which describes most Tesla models) are only involved in approximately 27% of all accidents. The Model X, a compact SUV, only accounts for about 13% of all accidents.
https://cdan.nhtsa.gov/tsftables/tsfar.htm# (look at table 37)
But it doesn't make for nearly as good of a press release.
Relevant xkcd: https://xkcd.com/1958/