The Math Behind the Rolling Shutter Phenomenon (2014) | Hacker News Reader

The Math Behind the Rolling Shutter Phenomenon (2014)

petapixel.com

87 pointsreddotX21 comments

Original submission (with 41 comments) from 6 years ago, with the original URL: https://news.ycombinator.com/item?id=8446932

Also, people should notice the "update" mention and shoutout to another blog post by Daniel Walsh, which went further into the math (and so parts were taken from Daniel's post to build the update section).

http://danielwalsh.tumblr.com/post/54400376441/playing-detec...

CMOS cameras almost always have a rolling shutter, but not always. At the Very High End of things, there are models that don't. e.g. the Sony PMW-F55 with a CMOS global shutter [1], around USD$30k

In the struggling camera industry, I bet a consumer-priced global shutter (or excellent computational correction for a mechanical/electronic roller shutter) could be a boost. Smaller, solid state, etc. The next logical step after mirrorless?

[1] https://pro.sony/ue_US/products/digital-cinema-cameras/pmw-f...

So here's a fun little hack you can do at home. If you have a signal (up to ~20 khz) and you feed it into an LED and look at the light through your camera phone, you can see the signal as bright and dark rows in the image. The bars are always in the same orientation (dependent on how your camera reads the data) and the spacing depends only on the frequency. One idea I've been toying with is using these properties to quickly isolate the parts of an image that are near to a transmitting LED in order to rig a camera into a 3d input device with very minimal image processing.

> If you have a signal (up to ~20 khz) and you feed it into an LED and look at the light through your camera phone, you can see the signal as bright and dark rows in the image.

Can you share an example photo?

I thought my phone camera was broken. It looks like the middle right side photo here, imagine in white/black overlaid on top of your photo with 50% transparency.

https://www.getpaint.net/doc/latest/images/tools/fill/Gradie...

Check your settings. If the bars are that big it is probably florescent lightbulbs running on 50Hz or 60Hz AC power. Most phone camera software quietly filters out the bars at this size/frequency. Another notable source is the 60-120Hz refresh rate of a computer screen.

Sure. Here's a direct photo of an LED blinking at ~1Khz https://imgur.com/a/Kmx9Qhe and here's one at 10Khz (you need to zoom in a bit) https://imgur.com/a/Q7oqs9J

When the light is passive (looking at the wall instead of the LED itself) the pattern is less obvious to the naked eye. Possibly invisible if there's enough ambient light. Around 20Khz the separation between bars is 1 pixel (on my camera) so you need to zoom all the way in to see it.

BTW check you phone camera settings. Most phones intentionally correct for the bars around 50/60 Hz because that is the frequency of AC power so florescent lamps will cause the artifact. The effect is usually unwanted.

Thats why my philips hues make that pattern. Thank for the info

One of the engineering teams I work with is building an image pipeline to correct rolling shutter distortion from a robot camera. I still have no idea how they're generalizing a solution to different types of movement and angular velocities, but at least now I understand the problem and a taste of the math behind it.

Thanks for sharing!

I've wondered if rolling shutter could be (ab)used as a poor man's one-dimensional high speed camera. Depending on the framerate, resolution and exposure time, wouldn't you be able to get hundreds of thousands of frames per second, at the cost of having no vertical resolution.

I'm not sure what it'd still be useful for with those limitations. Maybe estimating the speed of fast moving objects, or global phenomena like vibrations.

Definitely. There are programs for the Raspberry Pi cameras which do just that. Up to 1000fps IIRC

I don't think they really make use of rolling shutter, they just configure the camera differently? (which is basically limited by the rate of pixels transferred - if you reduce the resolution, more images per second fit down the data transfer channel)

I think they make use of the fact that individual lines are updated on the camera by the light and can subsequently be read out. If it were a camera with global shutter, chances are it could not be configured to update lines as fast indivually. A global shutter design copies each pixelsite or line into an indiviual buffer when the "shutter" signal is triggered. This happens in parallell.

I'd say these high framerates on Raspi cameras definitely is an artefact of the rolling shutter design in these cameras.

Maybe one could get even higher framerates by hooking the analog part (or very close thereafter) of the rolling shutter camera up to something else, but I don't see how.

Correct, a CMOS sensor with a rolling shutter looks and handles a lot like a piece of DRAM. Pixels are arranged in rows and colums; capturing a frame works just like a sequential read through a DRAM array. The nominal frame-rate at a given resolution is set by the read-out speed of the array, so reducing resolution increases speed almost linearly.

OT: Ugh. Another site that hijacks the back button to display more articles rather than letting me go back. (Chrome iOS) When did this start?

Edit: Video: https://share.icloud.com/photos/0qLMMePqiSOETXWt2Z77Ae8jA

Certain types of mechanical shutters exhibit the same phenomenon at high speeds.

See: Jacques Henri Lartigue’s picture of the Grand Prix of the Automobile Club of France, taken in 1912. You may have seen the photo before. Notice how the car leans forward and the spectators lean backwards.

https://www.holdenluntz.com/artists/jacques-henri-lartigue/

> Certain types of mechanical shutters exhibit the same phenomenon at high speeds.

Pretty much all focal-plane shutters do it beyond 1/60 to 1/250 of a second (identical to the flash sync speed), because the finite movement speed of the upper and lower curtains would result in an uneven exposure otherwise.

A related video from Matt Parker, aka Stand-up Maths, on this phenomenon https://www.youtube.com/watch?v=nP1elMR5qjc

I suppose it's coincidence that some of the examples look like relativistic stuff round black holes.

Eh, this is from 2014.

Added the year now. Thanks!

Older posts are fine.

Reply on news.ycombinator.com