Every department I've worked in mandates 2 supervisors to mitigate against this, because it's reasonably common for people to move departments, go on sabbatical or just quit. Even with tenure, life happens and people need to leave their jobs. In theory, the department shouldn't allow advisors to take on more students if they're close to retirement though.
Buddy of mine, his advisor died in like year 4.
Turkish university system solved that problem by stipulating that thesis supervision and advice duties survive retirement unless the student explicitly requests otherwise, and end only if the student finishes or the supervisor/advisor is fired (not resigned!) or dead.
Or your advisor accepts a job at a company in another state in year 2...