Fully minified and gzip compressed I've found SVG to be neither smaller nor larger than an alternate PNG version. It really details on the complexity with the image itself. Obviously you get the scalability factor for free. I suspect mutliple SVG files inline of the same document will benefit for a small compression boost.

Sending them inline of HTML simply reduces HTTP requests. Latency there is a big bottleneck. You have to balance that with the fact that the browser can't cache the SVG for use across multiple pages, like it can with a background-image or <img> element.