One thing I've noticed seeing the Lexus cars drive around Mountain View is that they're incredibly cautious.
I wonder what will happen once they become more commonplace, and human drivers realize they can be incredibly aggressive around the self-driving cars - cut them off, etc - and the self driving cars will happily accommodate the human driver, except with better reaction time and precision than a human driver could ever have.