The virtual frame technique: ultrafast imaging with any camera

Simplified summary; recording high speed video is all about compromise. Traditional high speed cameras trade resolution for extra frames. This technique trades the bit depth of the sensor for temporal resolution (extra frames). So a 16 bit sensor image is transformed into a monochrome (1 bit black and white) video. This is sufficient for studying high contrast phenomenon like crack propagation and wetting.

I wonder if you could use machine learning to “colourise” such a video. Perhaps by combining it with a slower frame rate full bit depth video and let it fill in the gaps.

As far as I can tell what they're doing is introducing the assumption that the subject's evolution is "binary and monotonic" - that is, the process you're interested in consists of things going from "light" to "dark" (or vice versa), in one direction only. They note that you can extract the propagation of such a process within a single frame merely by thresholding at multiple levels - since the process is "binary" the brightness of a pixel tells you the ratio of bright and dark temporal sub-frames, and since it's "monotonic" you know that all the "dark" (or vice versa) subframes are packed together at the end.

It's an interesting idea, but far from general purpose. I would say the paper is as much about the applications they managed to invent under such strict conditions, as it is about the technique itself.

I was hoping that the comments would have summarized the article.

Damn. I'll have to read it myself. :-(