You forgot that it also requires XML and CSS. I'm surprised PDF is just marginally bigger, considering that it's carrying around lots of legacy stuff is so much more than just a vector graphics format.
Anyway. SVG is not or only partly competing with the formats you listed. It's actually a format that's supposed to be included in those formats you listed. It's competing with PNG 81 Pages (http://www.w3.org/TR/REC-png.pdf) or EPS 31 Pages (http://partners.adobe.com/public/developer/en/ps/5002.EPSF_S... which is kind of cheated though because it requires the ~800 page postscript specification, at least it doesn't need XML and CSS additionally)
Removing sections that are standardized somewhere else doesn't make a file format easier, especially if you want to implement it outside of a browser.
There's still tons of features in SVG that are probably not used or not really needed:
Mandating IEE 754 binary floating point coordinates, while representing them as decimals that are usually rounded to 0.001 or 0.0001. This causes rounding errors all the time. The 16.16 fixed point format used in ttf seems more sensible for this usecase.
Including (parts of) external SVGs with <use>. <use> doesn't seem to be supported in Safari, Firefox or Inkscape anyway.
<switch> to make an SVG render differently on computers with different locale.
ARIA (accessibility) support, what happened to a description of the image?
Commas as optional separators in paths, but only at certain locations and only at most one of them. There have been several bugs in programs using SVG concerning the parsing path strings that were related to white space.
Basic shapes that can just as easily can be constructed using <path>, especially the polygon and polyline.
The total absurdity that is <text>. Text usually isn't the main content of images, yet the chapter about text is the longest in the SVG specification, containing stuff like text flowing along a path. And all this without supporting multi-line text. Just use <path> instead, bitmap graphics also uses rasterized text. If fonts aren't installed it may cause overlapping, or it might not get rendered at all.