A major use-case is to efficiently play a clip from a video camera, so purely vector animation or a sequence of frames from still images isn't good enough.
I've tested an SVG with an <svg:video> element (http://www.w3.org/TR/SVGTiny12/multimedia.html#restrictedVid...), but unfortunately it doesn't seem to be supported anywhere.
I've also tested an SVG playing video via HTML in a <foreignObject>. In Firefox it does play when the SVG is opened as a standalone file, but not when embedded in <img src>. So close :(