The justification in the HTML case is that "view source is good" and "it's compressed over the wire anyway". Don't those arguments apply equally (or nearly equally) well to SVG?
I've both created and modified SVG documents by hand with a text editor, and also view-sourced SVGs to see how they work.
The only thing that is especially unreadable are the long strings of coordinates that make up paths, and even those can be manageable with comments. No one does complex paths by hand, but the rest is colors, shapes, transforms -- all things you can read and may want to modify.
I used SVG as a templating/markup language for years. We were writing software for a specialized, embedded, browser that only understood JS and SVG.
It only took a few days of acclimatization to move from writing HTML to writing SVG.