That sounds like a workable model for vector graphics in browsers; it's just not describing SVG's design. It very deliberately was meant to contain not just vector model, but also a dynamic/interactive environment.
From the standard: https://www.w3.org/TR/SVG11/intro.html
"Sophisticated applications of SVG are possible by use of a supplemental scripting language which accesses SVG Document Object Model (DOM), which provides complete access to all elements, attributes and properties. A rich set of event handlers such as ‘onmouseover’ and ‘onclick’ can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards, features like scripting can be done on XHTML and SVG elements simultaneously within the same Web page."