This was already tried to some extent with Flash. Many sites were a single Flash module (or multiple). Implementing accessibility then required maintaining an invisible DOM representation of the Flash canvas. I personally don't want to return to the era where login and register buttons were dedicated Flash modules (MegaUpload did this back in ~2009) and many sites were an 800x600 image using the <area> tag for hitpoints.
Even Flutter has a DOM target because painting to a full <canvas> violates at the very least the operating system's text rendering, scrolling, and accessibility.