So how did web fonts end up as the approach? Because that's where browsers first acquired the notion of confluencing semantics and vector graphics. Fonts demand all the advantages of those entities: semantic information to define characters suitable for mapping to glyphs, the ability to apply styling to those glyphs (bold/italic/etc), and raster-independent scalability.
Could SVG do something like this, include semantic hooks in a file for applying styles? (I'm not too familiar with the format.)