I agree. When I initiate an action like "delete" and a dialog pops up, my mind is already assuming it's asking the question "Are you sure?" regardless of what the text says, and my eyes are just scanning for a "Yes" or "No". I would give preference to it being on the left (probably because "yes or no" is so ingrained... when's the last time you heard someone say "Answer me, it's a simple no or yes answer!"), but, realistically, if the buttons are close together I'm reading them both at the same time while my eyes are still moving toward them. As in many reading contexts, my brain knows the shape and length of the words I'm expecting so I'm mostly on autopilot until the very last moment.
If the action is to move forward with something or not, I think I prefer the "next"-type button to be on the right, and "cancel" or "back" on the left, probably far left.
But, my thinking does violate the "pick one and be consistent" suggestion. So... I dunno :)