I'm not talking about situations with two vehicles on the road. What happens with regular traffic is that you get someone driving 75mph in the right lane and another driving 75mph in the left lane, and it's game over for the whole highway. Once a few more vehicles accumulate, there's no safe driving stance. If you stay in the left, a gap develops on the right and people coming up to the jam pass to the right and then unsafely merge in. If you stay on the right, people passing others on the right are closing on you at ridiculous speed, or you're stuck driving a couple of feet away from another vehicle (on your left) that's tailgating and being tailgated and jockeying for position.
Taking the stance of "I'm traveling the speed limit, so I can stay in the left lane as long as I want" ignores the reality that there are other drivers that don't necessarily agree with you. Staying right except to pass minimizes the risk to everyone involved, accommodating both conservative drivers and aggressive or unsafe drivers, and lets everybody coexist. Staying left increases the danger and promotes road rage and other unsafe driving behaviors such as tailgating and right-lane passing.