An Algorithm That Unscrambles Fractured Images

This reminds me of dual photography: http://graphics.stanford.edu/papers/dual_photography/ in the sense that I imagine unscrambling the glittered image will be significantly easier if you can plug in known sources to the transfer matrix.

Edit: the video there didn't work for me any more, here's the youtube link https://www.youtube.com/watch?v=p5_tpq5ejFQ

It has occurred to me a number of times that it might be possible to reverse this process and create a light field display.