I'll typically just sit there until the person yielding their right-of-way gives up and goes. No way I'm pulling in front of someone with the right-of-way only to have them change their mind on a whim. I just assume they're trying to commit insurance fraud and ruin my day, whether they are or not.