Different patients need differing amount of times. We're not talking about doctors who work in private practice and have business hours. We're talking about doctors who work in hospitals and work prearranged shifts and have on-call time.
A doctor can't necessarily predict how much time a patient will need. If a doc is on an 8-hour shift (hypothetical; I doubt any docs are so lucky!), and gets a new patient at the 6 hour mark, it might not be known if the patient will only be in the hospital for an hour, which would be fine, or 4 hours, which would push the doc to 10 hours.
As the argument goes, that patient is safer staying with the doc into his/her 10th hour on the job, versus being transferred to a different, fresher doctor midway through. I think there's enough truth there for it to be persuasive, but 1) there are limits to how effective a doctor is going to be after a certain amount of time, and the benefits of patient continuity must start dropping as the doctor has been working longer, and 2) there seems to be little attention paid to improving the process of handing a patient off between two doctors, which could further reduce problems related to lack of continuity to the point where a doctor who has been working 10 hours will cause more bad outcomes than shifting patients to fresher doctors would.