Because in a busy environment (where consistently more than one person uses the elevator) the scenario of making a wrong decision and having the wish to correct it occurs far less than the scenario where someone does not pay attention to which floors have been selected already and deselects your destination. At the worst case this means that you both miss your stop because you both did not notice.