Whats wrong with SVG? Doesn't it accomplish what the OP wants to do already and is already a standard supported by a lot of applications? I must be missing something here
<!--...-->
</points>heh
So you could possibly argue that XML (a la SVG) is the right format to represent the vector files within that container, but it wouldn't be appropriate for the container file itself since binary data really bloats XML.
<svg viewBox="## ## ## ##">
<image xlink:href="layer1.png" x="##" y="##" width="##" height="##"/>
<image xlink:href="layer2.jpg" x="##" y="##" width="##" height="##"/>
<rect x="##" y="##" width="##" height="##" fill="red"/>
</svg>
All that's left is to standardize a way to package it.