As my driving instructor used to say, "Stop signs and red lights do not actually reach out and stop the other driver."
The lesson: just because it's your turn to go does not mean that others will not enter the intersection when it's not their turn. I agree that rolling stops at a 4-way stop are generally safe when you can clearly see that no other cars are at or approaching the intersection. But in my experience, the fact that the other person is at fault has been little consolation for having to deal with a totaled car and a permanent backache.