Hacker News Reader

Comment by ajross | Hacker News Reader

What I don't understand isn't the design of the lenses, which are chasing ever-tightening windows of precision in a competetive market with high margins and discriminating consumers. Of course they're going to be overdesigned.

What gets me is that lens distortion is a fully reversible convolution. It's not lossy, at all. Or rather, it's only lossy in the sense that the existing quantization in the sensor is already lossy: pixel boundaries, spectrum of the color sensors, etc... all have straightforward analogs in both space and frequency domain.

Basically: why hasn't someone come out with a camera with a junky single-piece lens and a fancy image processing backend and won the market?

View on HN

You're not capturing the rays, you only get where they strike. You can't fix it all computationally.

Secondly, we already do correct distortion and chromatic aberration in software, but this doesn't really make up for loss of resolution that you can't fix. Information is being lost.

Isn't the location that a ray strikes entirely a function of angle of incidence, its frequency, and the particular lens? Why couldn't that function be determined and inverted?

Optimally yes. But in practice multiple rays from multiple angles strike the same sensor.

Lens design is about minimizing this.

The phase matters as well if some parts are captured out of the focal plane.

Even if reconstruction is possible, it might amplify the noise by a substantial amount.

That's the point though: the quantization absolutely introduces errors, but there were there to begin with. You're comparing the computed result of my "cheap camera" not with its competitor's sensor output but with the hypothetical focal plane of the competitor's lens. And that's wrong, because the competitor has pixels too.

The error you get out is of the order of the sampling error you put in. That's true of all lenses.

I'm saying with the same sensor, the competition will have better data compared to you, since the rays are striking where they should be.

Information is lost when a ray strikes somewhere it shouldn't. Things get blurred and cannot be recovered. You can't fix this with software.

> You can't fix it all computationally.

Can't you? If you're dealing with known hardware, then you can apply a deconvolution. Blurring can literally be undone, people do it all the time. It's harder when you have to estimate the kernel, but much easier when you already know the lens's exact properties. Also when you apply it directly to RAW data.

So not exactly sure which information you're referring to you when say information is being lost?

Sure artifacts can be introduced from noise, etc., but that's all just tradeoffs. If a simpler lens is letting in more light, or you put money towards the sensor rather than the lens, the end result may well be better, no?

Some blurring is like pixelization

You don't have the exact kernel, if you're even slightly wrong you can end off being worse off. You'll have very significant variation in lower quality optics.

And there's a variety of sharpening tools that claim to do this! DxO is a company that sells a raw processor for precisely this purpose. People still buy better lenses.

I think you might be able to do a good job if you had a depth map of the image. You could use ML to guess them. Computation without that treats incident angles that vary by the distance ratios as the same.

If you have depth information blurring can be undone theoretically, but you don't.

What a lens does is fully reversible if you have phases as well as amplitudes at infinitely high resolution.

Unfortunately, you don't. You have a sensor that captures only amplitudes, on a spatially quantized grid, with some noise. It loses a lot of information. If your lens has very low aberration and not too much distortion, most of that information is information you don't care about. If your lens has a lot of aberration and distortion, a lot of the information you care about is encoded in exactly the stuff that the sensor is losing.

But of course they have - every major smartphone manufacturer is eating the whole consumer camera market alive for quite few years now.

Nobody has done this with a full frame sensor, as far as I know. I'd be interested to see smartphone image processing with a sensor that size.

There is fairly new Zeiss camera that implemented the whole "camera OS" with Android and even included mobile Lightroom editing capabilities. I guess it's as close as it gets right now. My theory is that the main Japanese companies are extremely conservative and they don't pick up such changes easily - see how Canon and Nikon completely missed mirrorless and suffered huge market loss to Sony.

Nikon made a mirrorless camera series (Nikon 1) that was very capable (the on-sensor AF was great). But they chose a 1" sensor (2.7x crop)

Canon's M-series was similar, they didn't go all in on "pro" features, instead focussed on a consumer market that was shrinking rapidly.

The problem wasn't the technology, the problem was the fear of cannibalizing their own DSLR sales.

Most image processing has to do with imitating full frame sensors (better low light photography, DOF, etc.) So I'm not sure if full frame/APSC/Med-format cameras need the same image processing that camera phones have. Also, most people who use the bigger sensor cameras are probably pros. Most of them would prefer a "blank slate" and use RAWs.

Camera phones have significantly better processing than newer full frame cameras at this point; obviously they can't take the same pictures, but full frame cameras can't do HDR, noise stacking, night mode etc very well. You can do it by hand by shooting raws and editing them together but it's inconvenient. The autofocus is also more advanced in iPhones with lidar.

I assume part of this is that the camera companies are Japanese and refuse to pay any engineers more than $20k a year.

My Olympus does in-body HDR, noise reduction with stacking, low-light handheld with sensor stabilization, and more. It's not "full-frame" but full-frame is a meaningless condescension from the 35mm crowd.

Full frame sensors are quite useful indoors because you can shoot 50mm-e without having to be twice as far away.

For real pretension you have to get a Leica.

Besides that, the main problem with the processing in those cameras is they don't do it to RAW, only to 8-bit JPG, so nothing for HDR displays. iPhone can do fused RAW, 10-bit HEIC, and takes HDR photos and has an HDR display to show them.

It looks like they still don't really plan to add it: https://www.dpreview.com/interviews/9195902169/interview-son...

but at least if they had better bracketing options we could do it ourselves.

yes, but perhaps this is because of other things? Like that a person is going to have their phone with them anyway? Even if a phone camera is only 1/2 as good as a consumer camera, the ease of use and convenience is going to win the 99% of the market which doesn't care about 'professional grade' results. And the 1% that does care is going to buy a SLR.

Not better tech (better as in takes better photos), rather more convenient tech which is good enough.

Good enough for most people's point-and-shoot, but a smartphone camera is by no means competitive for anything beyond that. Hobby photography is fairly common too, and most people won't be using smartphones for that.

You are not quite correct. The deconvolution only works well if you know exactly the convolution kernel and you have infinite signal to noise. In practice neither is true and you'll have artefacts if you try to do deconvolve. And the reason why it's hard to know exactly the convolution kernel is that it will depend on the wavelength of the photons, position in the focal plane and potentially orientation of the camera, if optical elements are moving around.

A junky single-piece lens won't do it, you can't correct everything in post-processing. Information that is lost is lost. But there are corrections which can be well made computationally, like lens distortion. As a consequence, there are several modern lenses which trade more distortion for general improved sharpness. The distortion then gets corrected digitally. As a consequence, you do get improved picture quality vs. what would have been possible in the same package otherwise.

Isn't that exactly what modern smartphones are doing? I think it's quite impressive what Apple and Google are able to do with clever image processing.

What GCam does is simple stacking to improve snr plus some clever alignment/de-ghosting to avoid motion artifacts. It mostly compensates for the sensor size, not the lens simplicity. They still have many artifacts which big lenses don't, even after software compensation calibrated to match the specific camera.

>Basically: why hasn't someone come out with a camera with a junky single-piece lens and a fancy image processing backend and won the market?

that's what phone cameras are...

And a lot of lens aberrations not correctable robustly. LCA is basically impossible to correct, field curvature is impossible to correct, bokeh quality is impossible to correct (within reason), wide depth of field is impossible to correct, dynamic range is impossible to correct (without bracketing), coma is impossible to correct, flaring is impossible to correct and the list goes on.

Camera phones have had multi-element lenses for years, typically around six.

That's basically what micro 4/3 cameras are doing. As far as I know they all can do distortion correction in-camera for JPEG output, and the prime lenses are generally quite cheap and very compact.

For example Panasonic has a 25mm f/1.7 lens that is 125 grams and costs $150 brand new.

Of course you can also buy similar focal length lenses that are huge and weigh over a pound, while spending $1500+. But I don't know how much advantage those really give except in specific rare situations.

I like m43 cameras, but the depth of field with that sensor size is nowhere near that of full frame

I disagree. The Panasonic 25mm f/1.7 is equivalent to a 50mm f3.4 on full frame.

For 150g on full-frame you can get a 50mm 1.8, at a cost of 150$ brand new. So on full frame at a similar weight and at the same price you get superior optics.

If you really care about the weight, the 40mm 2.8 has the same weight and the same price but superior optics again.

All modern FF cameras do distortion correction too. M4/3 lenses are not especially decided to trade-off distortion for sharpness anymore than full-frame.

Whether you should apply crop factor to aperture depends on whether you're interested depth of field, or exposure. You may or may not be able to adjust ISO to get comparable results at 50mm f/3.4 depending on which cameras you're comparing.

If you adjust ISO you will get more noise.

Noise is decided by the quality of the sensor, and the solid angle of light that hits each point on the sensor. The solid angle as it happens also decides depth of field.

So it is always true that you should apply crop factor to aperture. ISO is just not the correct factor to describe gain because an inverse crop factor is applied to it in effect.

So at the same exposure and shutter speed, at f/3.4 on FF you will get the same image with the same amount of noise as you would at f/1.7 on M4/3.

You'll have to set ISO higher to get the same image at f/3.4 as you would at f/1.7 on a smaller sensor because the intensity of the light is reduced.

I'm interested to learn more about the effect of light angle on noise. Do you have a source that talks about this? I didn't find anything useful in 2 minutes of searching the web.

No, the intensity of light is not reduced. ISO is not gain. In other words, ISO 200 on full frame is the same as ISO 100 on M4/3 in terms of what actually happens inside the sensor. This is because of how the ISO standard is written.

As for light angle, it's a consequence of the second law of thermodynamics, that translates in optics as the conservation of étendue.

Basically what this means it that the amount of light captured from a light source is equivalent to the solid angle of light from that light source onto the entrance pupil - whose size is the focal length divided by the f number.

So basically, what determines in the same scene how much light is received by the sensor depends on the size of the aperture in milimiters.

So on a sensor of the same efficiency, it's the physical size of the aperture that determines how much light is captured.

The intensity of the light is reduced because the aperture is the same diameter (50/3.4 = 25/1.7), but the light has to cover four times the area. To get the same exposure with double the f number and the same shutter speed will require four times the ISO.

> what determines in the same scene how much light is received by the sensor depends on the size of the aperture in milimiters

That's true, but the amount of light is not the intensity of light.

It's the amount of light that decides the amount of noise.

Yes, you will need four times the ISO. But you will get exactly the same amount of noise.

I believe this was essentially the hope of "Lytro", or light-field cameras more generally.

https://en.wikipedia.org/wiki/Light-field_camera

That market — the one that would be happy with a mediocre lens and great image processing - likely overlaps with people using top tier mobile devices for their work. After all, acceptable lenses with amazing image processing is basically what the flagship iPhones and android devices bring to the table, and their use case tends to overlap with that of a 50mm prime.

This strikes me as true.

I wouldnt say the camera companies did not see this coming. They did and chose to not do anything about it. I shoot videos with some pretty low end DSLR cameras and they are often lacking the feautures of any modern flagship phone. It's a joke. We once used a whole Iphone shooting at 120fps and made some plain scenes look amazing in post. A revelation since most pro videographers pooh pooh phone cameras without realizing that they are growing with alarming speed.

The pros are well aware of the gains made by phone cameras. Sensors and pixels aren't the primary factor for choosing say, a $10,000 camera, over an iPhone. Reliability, battery life, form factor, control ergonomics, lens compatibility, codecs, ease of integration with third-party equipment, standards compliance, and on it goes. I don't think the opinion is that an iPhone can't make a nice image, just that they are woefully insufficient for many needs for reasons other than the sensor and processing algorithms.

Virtually every DSLR and mirrorless cameras will correct for distortion in camera. On Nikon it is called Auto Distortion Control, you can turn it on and off after the fact for raw images. Lightroom and most other photo developing software supports this as well, with profiles specific to each lens and focal length. Turning it on results in the image being cropped very slightly as a result of the image being "straightened out" in camera for Nikon, or with a slightly larger overall size otherwise. Besides that very noticeable change there are physical limits on what the software can do, a lens with less distortion will have more detail as a result

why hasn't someone come out with a camera with a junky single-piece lens and a fancy image processing backend and won the market?

I believe it’s called the iPhone.

Doesn't do much of any lens compensation. It compensates for the sensor quality, but not the lens. There is portrait mode of course.

Your intuition is correct but, unfortunately, it’s not simply a convolution.

A great example of this is DiffuserCam [1], which uses a diffuser as a “lens”, combined with a carefully measured point spread function (PSF) of a single point for each axial plane to estimate the “true” PSF that actually varies throughout the entire volume. So although in this work they estimate the reconstruction by making a translational invariance assumption, this is done for practical reasons and quality is lost.

To fully recover a lossless image as you suggest, one would have to measure a PSF for each point in the volume.

[1] https://waller-lab.github.io/DiffuserCam/

I would argue that this happened with smartphones. "Real" cameras are generally for people who want to chase that optical precision, but for people who are satisfied with a good-enough post-processing solution will generally be satisfied with their phones.

There are "Light field cameras" that try to capture lightfields / light flux. However, their ray angular resolution isn't anywhere enough to really do too much fun stuff.

There are also single-pixel compressive sensing cameras, which don't need focusing optics at all, so they can be made lensless (and potentially flat). They also have plenty of downsides though, such as poor performance in scenes with movement, or very high computational requirements for image reconstruction.

There are already manufacturers who do in-body correction alongside their lens design to get results that would require a ton more glass; Olympus are one. Sites like DPReview use it as a relentless talking point that those manufacturers are sub-par compared to the big advertisers.

It's the photography equivalent of "if you were a real audiophile you'd only use pure analogue signals, digital destroys the sound".

I think that market is largely already consumed by phones. For most people, there is no reason to not just use a phone to take picutres.

Once you are in the 2k$ usd lens space the buyer is looking for ultra premium stuff either as an enthusiast buyer or as some one who makes their living that way.

>What gets me is that lens distortion is a fully reversible convolution.

This is what I thought (it's not really convolution, but yes it is correctable), until I realized darktable/lensfun doesn't have lens profiles for any of my lenses... :(

If you have time and wish you can add them yourself:

https://pixls.us/articles/create-lens-calibration-data-for-l...

You'd need a curved sensor to get a flat focal plane if that's what's desired. I personally don't quite get it either but

Lots of m43 cameras do some of the corrections in software, which does keep lens sizes and prices down.

Not just m43s. Sony/Nikon/Canon, I'd assume the rest of the major brands as well nowadays; most frequently for Pincushion and Barrel distortion.

Such corrections are all but essential for affordable zooms, but I'm still split on the potential for over-use in primes. I'll point to the Sony 16/2.8 E as an example; the use of correction in JPEGs is pretty evident compared to shooting raw, and you can see the correction softening the corners.

Reply on news.ycombinator.com