Unfortunately, we can't exclude humans that drink, text, have anger issues etc. from driving on the road with you. At any point, the driver next to you could have a stroke, pass out from doing drugs, have a panic attack, etc.
But I think you're getting at a really interesting question. If the ratio of self-driving cars increases in your zip code does your insurance, as a human driver, decrease? I.E. is if safer to drive next to a self-driving car or a human driven car?