People always say this, and I always ask "but then doesn't he have to pay back the loan?" and I never get a good answer.
You never have to pay back the loan yourself. Just make small monthly payments and refinance until death. This is actually a pretty widely-exploited part of our tax system, for very high net worth people.