(Source: I'm in the VR / plenoptic space, knew a bunch of people at Lytro, some of whom are now at Google. Timelines and facts do not match this article's assertions.)
(Source: I'm in the VR / plenoptic space, knew a bunch of people at Lytro, some of whom are now at Google. Timelines and facts do not match this article's assertions.)
Phase detection is not the same thing at plenoptic; the article implies that phase detection operates in the same way as a microlens array. It doesn't; microlens arrays for plenoptic imaging have many pixels underneath, and perform different operations on them. This line:
"With only two pixels in one, there's still enough to calculate a fair optical depth of field map without having a second camera like everyone else."
This is categorically false. You need dozens of pixels underneath a microlens array to do digital refocusing; phase detection is not remotely the same thing.
Later, the author conflates microlens-based refocusing with camera-array based plenoptic imaging. These are wildly different disciplines; the work done in VR does not include microlens arrays anywhere. Not any. But they do include large arrays of cameras (see volumetric work from Microsoft HoloCapture, Intel's capture studio in Manhattan Beach, and my own company Visby's work — cameras at Radiant Images in LA... all arrays of cameras, no microlenses anywhere).
"Apparently, if you take only one central pixel from each cluster and build the image only from them, it won't be any different from one taken with a standard camera." This is false; taking the central pixel will give you an aliased image (or an image as though taken with a very, very high f-number lens, if the pixels correspond to the same zone of the aperture). To make an image as though you didn't use a microlens array, you sum all the pixels under the microlens to produce a macropixel. This is why light field cameras have lower effective resolution than traditional cameras. The remainder of the speculation about 'sneaky plenoptic JPEGs' has no basis in light field imaging; refocusing is not achieved by throwing away (or binning) pixels, it's done by summing different sets of pixels. Lytro founder Ren Ng's PhD thesis was about a way to do it faster by using the Fourier Slice Theorem. (Beautiful paper, if you have time to read it.)
Google did not "buy and kill Lytro." Lytro's assets were sold and Google bought some of them, but none of that tech made it into the Pixel line of cameras, period. The Google portrait mode is done using phase detect pixels as a cue for segmentation, as discussed in the article this blog links to. Nothing plenoptic required, and certainly no Lytro tech there. The blog post implies that the Pixel is using Lytro-like techniques rather than phase detect pixels to hint at segmentation for synthetic blur.
I have never heard of plenoptic cameras being used for image stabilization. In my personal experience (5 years of stabilizing cinema cameras for two different companies), virtually all stabilization issues derive from rotational blur, not translational movement. Your ability to stabilize translational movement would be limited to the aperture size, and translations of that magnitude have vanishingly small effects on the image unless the objects are very, very close to the lens (like macro). I have no idea where the blogger came up with this idea but there is no reference provided and I have never heard of it (nor would it work, for the above reasons).
The section "Fighting the Bayer Filter" miscasts the challenges of a Bayer pattern under a microlens; naively summing adjacent pixels under other microlenses would blur an image. You do get a boost to dynamic range from the synthesized images — you're binning a bunch of pixels to create your image, so you reduce noise as 1/root(n) in the number of samples. That could lead to better color in dark, noisy areas, but it will not increase the gamut of your sensor.
It goes on... but since this article was already off the front page by the time I posed my comments, this is probably enough.
(As an aside, Jacques — I really enjoy your writing!)