There's a reasonable argument to be made that making forward progress while remaining safe when intelligent agents are willing to play chicken requires a good theory of mind, but I can't formalize it. It's "just a hunch", but that seems like a Hard Problem to me and not necessarily something we'll completely solve soon. There's been work along these lines with statistical/ML methods, but my personal suspicion is that existing work would quickly fall apart in a truly hostile environment.
There's also the discussion around navigating non-roads, potholes, and other infrastructural issues that the industry hasn't solved yet. I don't think these are insurmountable, just difficult engineering issues.
While its quite proven that slow, chaotic, and unpredictable environments are so much safer. Notice how walking through an all directions intersection crossing is incredibly chaotic, and yet no one collides with anyone else, or even when they do, it results in basically no harm.
The "slow" part, for sure. Harm in a collision is largely a matter of kinetic energy.
The "chaotic and unpredictable" part, maybe not. From the statistics I've seen over the years, traffic-related death rates seem to be much higher in places with highly chaotic driving environments. For example, 140 annual traffic deaths per 100,000 vehicles in India vs. 14 in the USA.
I think the answer is going to be government rules because it's important that self-driving cars are predictable when there's no human driver you can have eye contact with in complex situations.