To move everything at once drag the the body of the segment (not in between). That will move everything from that point on.