You absolutely can make a website without any HTML, not even a scrap of invalid markup.

Just use SVG. It does not need to be embedded in HTML to work.