This result (the shape that can theoretically be reconstructed from its shadows) is called the Visual Hull. See http://pdf.aminer.org/000/293/374/the_visual_hull_of_curved_... for the original paper.

Awesome writeup and experiments. Especially like the crossed-eyes stereo image.