The asymmetry comes from only one of the twins experiencing a force causing it to accellerate at the point where it turns around to come back.