Extreme depth-of-field light field camera thanks to an extinct marine arthropod | Hacker News Reader

Extreme depth-of-field light field camera thanks to an extinct marine arthropod

image-sensors-world.blogspot.com

I'd be interested in seeing it perform in real world situations, specifically with respect to horizontally and vertically polarized light. I can see why they avoided it in the study, using circularly polarized light, but in the real world, it could be very helpful to the owner to help in navigation, as the polarization of the sky is anisotropic.

PS: You can manually create light fields. Take multiple photos of the same subject from slightly different locations. Then use hugin (the panorama stitching program) to align them on a given region, and merge them together. You can create a virtual focus plane of your choice.

Even after reading Wikipedia, I'm not sure I understand what a light field is. But this looks pretty interesting.

You can think of a normal lense like an extremely specific filter. It ensures light from a specific direction, and that direction only, arrives at the sensor. Some also ensure only light oscillating in a specific way goes through (polarization), or of a specific frequency (with color filters).

This filtering is necessary for us to produce useful data from the total chaos that is the actual light field - aka all the light bouncing and going in all directions at once - where ‘us’ are our normal human brains.

Other ways of handling it though if you can use different types of information processing.

An ideal camera captures the light rays passing through a pinhole. It generates a 2D image where each pixel corresponds to a ray, and the X and Y coords determine the direction of the ray.

Because every ray passes through the same pinhole point, you can't do things like change the depth-of-field or move the camera once the image is created. (An ideal pinhole camera has everything in focus)

A light-field camera captures the light rays passing through an entire area. This generates (in theory) a 4D image. Still 2 degrees of freedom for ray direction, but an extra 2 degrees for ray position.

Because the rays pass through different points, you can re-project a light field image to create 2D images with different depth of field, or different virtual camera positions.

A surprising aspect is that the lightfield merely appears to be 4D. If you stop and think about it, all of the information can be captured in the 2D plane of the aperture. So effectively, it is just a 2D image!

This is akin to a hologram, which looks truly 3D, with depth of field, refocusing, and the works, but is physically a 2D photographic plate. Literally just a sheet. The hologram captures the 2D slice through the EM field the same way that a lightfield captures a 2D slice through the aperture.

The key thing is that the phase information must be captured also, which in the case of holograms requires a resolution finer than the wavelength of light. Lightfield imaging does it differently, but the same concepts apply.

When you say hologram, are you referring to something like a holographic sticker, or a star trek hologram?

He's referring to not the star trek hologram, and not really the holographic sticker. This is one holographic plate illuminated by a laser, viewed from different viewpoints: https://en.wikipedia.org/wiki/Holography#/media/File:Holomou...

And it's not like those things with the bumpy lenses, that show you an image from a few chosen viewpoints, either. You can look at that mouse from a wide range of angles and distances.

Handwavy, short explanation: All the light arriving back from the mouse-- in different directions-- is encoded into the speckles on the plate. When a coherent source of light is used to illuminate the plate, again, the light exits with the same characteristics it had entering the plate.

The idea of having a "field" is that a 2d slice of it allows calculating light elsewhere from the field using optical propagation calculations. But this requires phase measurements in addition to intensity which is only what conventional cameras capture.

It could be easier to understand by looking the parts of a light field camera. Compared to a conventional camera, one type of light field camera has a micro-lens array just before the sensor. So this is like placing a Shack-Hartmann wavefront sensor over the image (apologies for another technical term, but that roughly explains the concept and is easily looked up, implementation details might be different though).

An ordinary photograph only stores intensities, so changing the focus given the recorded image is not possible using physical optics calculations (ML/AI algorithms might do the trick, but that's something else). Based on the image formed from each lens in the micro-lens array (i.e. the displacement of the focus from the sensor), it is possible to retrieve the wavefront or lightfield which allows subsequent optics calculations, like post-focusing.

The tradeoff is that each micro-lens in the array needs a decent number of pixels for analysis, so the effective 2d resolution is decreased as compared to using all the pixels.

I love this. I have been fascinated by Trilobites since a kid: they are from hundreds of millions ago, like..500...but still look like common woodlice. Its not clear how they found the compound-eye-lightfield thing from this post, but I might be just impatient/reading over it. Would love to have some woodlice lenses in my cameraphone :P

here is the underlying Nature article https://www.nature.com/articles/s41467-022-29568-y

"extinct marine arthropod" = trilobite

Why use one word when you can use 3 with less precision?

Let's put that quote into context shall we?

> An article titled "Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field" by Q. Fan et al. proposes a metalens design inspired by the bi-focal vision system of an extinct marine arthropod.

Oh look at that, "trilobyte" was already used in that same sentence, which is the first one in the blog post. How nice of the author to not presume that everyone knows what a trilobyte is, and give a three-word summary that clarifies it enough to make sense of the rest of the article. I suppose "this extinct marine arthropod" instead of "an extinct marine arthropod" might have been even better, but I'm not going to nitpick that.

When you get paid per word. And indirectly search engines incentivize longer texts with more keywords.

I'm sure you already know the answer to your question, but the answer to your question is that it's to make the subject of the article comprehendible to people who do not know what a trilobite is.

Reply on news.ycombinator.com