The cars could be subject to vandalism, or at the very least really aggressive driving. The predictability of the driving and the lack of a driver is going to result in weird interactions with people.