> An alternative design would have been to have built Wayland inside X.org and tunnel the new protocol over the existing protocol, and then migrate chunks of functionality one at a time

You are looking at it from the wrong direction, though. But otherwise, that’s pretty much what wayland does for backwards compatibility with xwayland — it wraps an x session into a window you can normally use from wayland.