Like how phonetic alphabets save space compared to ideograms by just “write the word how it sounds”, the little SVG-icode would just “write the letter how it’s drawn”
Think of all the symbols notetakers invent; ideographs without even phonology assigned to it.
Being as dynamic as flexible as human expression is hard.
Emojis have even taken on this property naturally. The high-5 is also the praying hands for instance. Culturally specific semantics are assigned to the variety of shapes, such as the eggplant and peach.
Insisting that this shouldn't happen is a losing battle against how humans construct written language. Good luck with that.