It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though.
It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Instead, we rely on weaker protections. We try to make known poisons hard to make, we try to track people who could make them, and we try to make it hard to deliver poison.
I believe the same will be true of adversarial examples for vision (and language) models. We can try to make them hard to make, hard to posses anonymously, and hard to deliver. I think this will be much easier with computer vision than with poison, so I'm not worried about it.
For example, consider the case of pasting a sticker on a speed limit sign that causes Teslas to swerve off the road. Governments should protect people from this in multiple ways, similarly to how they protect us from poison:
1. People who post these stickers should go to prison.
2. People who create and distribute these stickers knowing their purpose should go to prison.
3. Tesla should be civilly liable for cases where preventing such an incident was possible with known technology.
4. Roads should be modified over time to make it more difficult to do this attack.
I think some combination of the above would be enough to make society as comfortable with adversarial example risk as we are with poison risk.