Record-breaking camera keeps everything between 3 cm and 1.7 km in focus | Hacker News Reader

Record-breaking camera keeps everything between 3 cm and 1.7 km in focus

newatlas.com

108 pointsbsenftner42 comments

The gallery somehow defeats Firefox's "disable disabling pinch to zoom" setting, and it's not clear if the New Atlas article has the full size images anyway.

Here's the primary source, with links to full size images:

https://www.nature.com/articles/s41467-022-29568-y#Fig6

The university picture, in particular has some strange artifacts in the tree lines. It looks like the neural network is inventing scene details.

Also, the aberration figure for the dolls suggests it's doing some sort of object boundary detection to decide what distance each pixel should be at.

This seems to have the same problems focus stacking would have. I wonder if traditional signal processing could fix it.

>The gallery somehow defeats Firefox's "disable disabling pinch to zoom" setting

Happens with Chrome and Safari on iOS as well, but only when you're in the 'view images' mode after clicking on an image.

For me I honestly can't think of anything more hilariously disproportionate than my physiological response to disabled pinch-zoom. It's a whole body experience, starting with this weird cramping/tension in the back of my left hand, which is holding the phone, then a wave of cortisol/adrenaline cocktail washes over me...followed quickly by a stream of expletives and an attempt to crush the phone with my grip or some other destructive outlet.

It's wholly irrational and feels entirely limbic in nature, probably a sign of how deep the pinch-zoom gesture is engrained in the psyche.

The actual example images in the article look... bad?

Maybe someone who understands optics better could help explain why they look unnatural. Is it an uncanny-valley analogue?

I believe what you are referring to is aberrations? https://expertphotography.com/lens-aberrations-photography/

Uncanny valley is more your mind not understanding what is being perceived, this is more that the image is not as sharp due to imperfections in the lens.

If the lens were perfect it would end up looking like any other picture you see with everything in focus. This is achieved today by stacking say, 3 pictures on top of each other where each picture has a different part of the subject in focus

That link took me to a description of how women should be posed with an "S-shaped figure".

I get what you’re saying about uncanny valley. My first thought when I saw the example image was “maybe it looks weird because my brain expects some parts to be out-of-focus?”

I have no idea if that’s true or if it’s just a bad looking picture. The sky looks weird.

It's reconstructed from sub-images, the reconstruction algorithm can probably be improved.

The lack of focal blur prevents depth perception, and the objects don't have "natural scale" clues (like a human or a car) so the photo looks like a flat 2-D collage.

We need eye tracking that would blur the objects that would normally be out of focus for what the person is looking at.

My opinion is that finished images from professional photographers are cleaned up interpretations. And increasingly so on an automated basis from smart phone image processing stacks.

Raw images often don't look as good to the eye, even if they contain fairly complete information that a professional or automation can work with to provide a number of different finishing approaches that are pleasing to human perceptions.

This techs raw capture is significantly different from current image sensor data, and their processed image is mostly making it close to a mature "raw" image is sort of how I take the sample photos in the article.

I agree with you about modern phone cameras, but I have an old digital with minimal postprocessing, and the images come out clear (at a whopping 3 megapixels), as do the raws on my Sony A7. Similarly, film cameras worked fine without piles of neural networks, etc during development.

The images in the article are all blurry, but seem decent for a research prototype. Presumably, dumping lots of commercial development resources into the technology will improve it.

I think I disagree with your opinion. Raw images often look fantastic if they are captured properly (in focus, exposed correctly). I still work regularly with old film cameras, and the photos develop very very nicely without any digital alterations. When I shoot with my digital I usually save both RAW and JPEG formats so I have the extra raw data if I need it but if I've taken a good shot I minimize my digital alterations.

There are some additional tweaks that can be done to a "raw" format camera image that contains extra image data compared to a processed/jpeg type flatter image, but there's no reason they should look bad coming out of the camera. Maybe unbalanced white color values at most.

Raw camera data requires an interpretation to be displayed on a monitor at all and usually needs a lot of processing to produce a natural-looking photo on the screen. Demosaicing at the very least but gamma curve correction, color response correction, etc. as well.

We rely a LOT on focus to get depth cues from 2D images.

When everything is made to be in focus, we lose depth cues and everything looks wrong.

That said, they certainly could have created better example images. Like, I would have placed an object like a figurine on an outdoor table with a city skyline in the background and taken a picture of the object from about a foot away. Normally, this would put the object in focus with the city being blurred, as well as part of the table. It would have made it very clear what they were going for.

Focus stacking program: "Look at what they need to mimic a fraction of our power."

While handy, focus stacking requires taking multiple images which means you're limited to static subjects only. If I'm not mistaken, the proposed new approach can just as well be used to take pictures of moving subjects.

I wonder if instantaneous focus stacking could be possible with a module of cameras.

Yeah if you had multiple sensor outputs with different known fixed focuses it could likely be done on the fly.

The limiting factor here is that you need your lens to shift focus. So if you wanted to take a-la five simultaneous shots for focus stacking you would need five separate lenses at different focal distances. Since five lenses can't occupy the same physical space you're going to end up with five slightly different perspectives. It might be possible to computationally correct the perspective between them to a certain extent, but I'm not sure how good the end result could theoretically be.

It could also be done through partial mirrors and prisms, but would of course require a brightly lit field or suffer ISO tradeoffs

I'd say the meme should be reversed - focus stacking requires multiple photographs, while this one only requires one.

How is this any different than taking multiple pictures at different focal lengths? The fancy optics split near and far objects towards different areas of the sensor. Why bother? Why not just use two or more lens paths each aimed at different parts of the sensor? Or use multiple sensors?

Your proposals increases physical complexity(think manufacturing price increases) and therefore more points of failure.

The seems sort of like what Lytro was doing, just taken one step further. Instead of focusing the pictures after the fact, they take the light field and combine the different focus levels to produce one fully in-focus picture.

> combine the different focus levels to produce one fully in-focus picture.

That's not the novel. part. Everyone's phone and DSLR with focus stacking does this.

The novel part is achieving a deeper range of depth of field.

Lytro could also produce a fully in-focus picture. And a light field isn't made of "different focus levels", but rather the reverse - you can generate different virtual focal planes using the lightfield (Lytro's gimmick).

I'm still waiting for the day when the entire back of every cellphone is a flat compound eye surface lightfield camera.

It has proven much cheaper to guess all the extra information rather than capture them.

Well, sure, _for now_.

To assert that that will be true for the future is just a failure of imagination.

It's like the assertion that "640k should be enough for anyone".

I want my 5gb per snapshot light field information! :)

It seems to me, based off of their sample picture that the advantage would not be so much in getting one single frame where everything is in-focus. Instead I would be more excited about being able to "change" the focus after the photo is taken. I assume that would be equally possible with this camera (depending on how the raw data is stored).

> Instead I would be more excited about being able to "change" the focus after the photo is taken.

Lytro was doing that in 2012: https://en.wikipedia.org/wiki/Lytro

For artistic purposes, changing the focus may make sense indeed.

For most other (e.g. sensing) applications ... why would you want to ruin your data by applying a convolution to it and losing information?

About a decade ago, I had one of those rectangular Lytro cameras. https://en.wikipedia.org/wiki/Lytro

And yes, you could change the focus on a picture that was hosted on the web.

Unfortunately, those photos, as they required the Lytro site to work, no longer work and I can't find any working archives of them. ( Lytro is winding down https://news.ycombinator.com/item?id=16705391 )

But yes, you could - the Lytro's first camera section of https://youtu.be/m8c1__yzX2Q showed some of the refocusing.

I got a Lytro ILLUM to test before they began selling it in Spain years ago. Seemed like an interesting toy, specially the depth map that was unusual for that time (now there are cellphones capturing depth maps with different techniques).

If I check my NAS there's the dump of the SD card I used that day, but I wouldn't know how to open those files or process them, as I guess the software is not available anymore and I don't know if anybody has hacked the file format.

On the Lytro, it had a filesystem with an image viewer for the raw data in the camera itself (plug it in and there's a disk that shows up). I'm not sure if the Illum had a similar tool for manipulating the raw data that was part of the camera that was sent.

"Trilobyte", heh. That's a funny typo.

How is this different from the Lytro?

This camera has larger depth of field.

The article explains the tech, but I don't know how much of Lytro's tech is similar.

Both are "light field".

From a quick glance they look quite similar in its basic concept. I'd say the main difference is in the metalens that splits the image, that seems quite more advanced in this prototype.

I guess the CSI meme about "Enhance, Enhance, Enhance" will no longer be a meme but reality.

Killer robots get yet another major advantage.

Isn't it what Litro was doing back in the day? A friend had one, it was very impressive. The form factor was weird though.

Reply on news.ycombinator.com