Modern smartphone lenses are crazy

Thank you for the links. How did you originally find yourself falling down this rabbit hole?

The craziness in the iPhone design isn't exactly the number of elements, but their weird aspherical shapes. The traditional approaches for designing optics are have been developed for spherical lenses because they are much simpler to describe mathematically. Even then, optimizing them requires a lot of numerical computation. At first glance, optimizing these wavy, complex lens shapes seems to require much more involved methods.

Interesting. He doesn't really explain how they're designed though. Must be just automatic global optimisation at this point though. No way a human manually optimised all the ray paths.

Boy, that page is Comprehensive on lens design - that's an awesome resource.

>The name “Anastigmat” literally means “non-astigmatic” lens, since it corrects astigmatism and its cousin the field curvature. Which is funny, because astigmatism means “non-stigmatic”), where stigmatism is “an image-formation property of an optical system which focuses a single point source in object space into a single point in image space”.

>I love the word Anastigmat. That means that “Anastigmat” can be translated as a “non-non-stigmatic”. I guess “Stigmat” didn’t really have a good ring to it. Nowadays, almost all lenses have the basic aberrations corrected, so we don’t have the need to call them Anastigmats anymore. Sad.

I think the lensmaker's equation is pretty par for the course for most advanced physics classes in high school, no?

I saw it in A level physics, and it was part of the standard curriculum, so not that rare.

In high school it’s introduced when they close the blinds and turn off the lights and you play with the glass blocks and 3 lines of light, if that jogs your memory.

I think its the former, if you are in the US. I also did not learn this formula in high school. My physics class barely went beyond momentum, and I remember the slightest hint of electricity being discussed.

High schoolers probably learned the thin lens equation.

Correction: It was released 2016, that's 6 years ago.

The point still stands though. But 6 instead of 8 years.

I don't think there are any really big changes to the lens design recently... All the effort has gone into multi-camera setups, and different types of lens for macro, wide angle, telephoto, etc.

To my understanding, there is still only a single set of moving lenses in an iPhone 13 lens.

The iPhone 12 Pro is insanely better than the iPhone X--especially in low light. I have a bunch of fairly high-end camera equipment that I rarely use outside of some fairly specific circumstances at this point

I suspect that's mostly down to sensor design (big pixels, reflective layers behind the sensor so the light goes through twice, etc) and software.

If I remember right, newer Samsungs have their stack of telephoto optics pointing toward the top of the phone, parallel to the display. Then use a periscope style prism to look out and forward. That way you aren't limited by the device thickness (although I guess now the diameter of the lenses is constrained)

Polynomial models are generally usable for most applications. If you need lots of accuracy, such as for long-range stereo, they sorta work only for traditional and non-wide lenses.

From what I've learn from 3d software tutorials (trying to simulate a glare), this glare comes from the lenses and each lens will add another ring in the glare? not sure.

Maybe dumb to suggest, but when this happens usually the lens is dirty / a bit greasy, at least from my experience with iPhone cameras.

The OnePlus 3 had a sapphire camera lens, Apple does not. Maybe that's why the flare.

Plastic typically has much worse reflection and losses than AR coated glass (at least for narrowband lenses - I'm extrapolating to photo lenses).

I think the main reason to use plastic would be the potentially higher refractive index and better strength properties, which are also why plastic lenses are used in spectacles (some glasses can have higher refractive indexes but still require heavier thicker lenses to avoid the risk of shattering). I would assume that glass lenses would be more likely to shatter if the phone were dropped. But it may really be that the plastic lenses can be manufactured to a higher quality for a reasonable cost.

In my experience, though I can't speak too much about commercial lenses for DSLR or even SLR camera systems, but can speak to some degree with regards to something like a payloads used for aerospace and defense applications.

Designs of aspheres are typically created via Zernike Polynomials. If you have access to an Optical Design program such as Code V, Zemax, OSLO, ASAP, HEXAGON (Raytheon), or Quadoa or equivalent one can spit out something called a SAG Table which takes in account the Zernike Polynomials and allows for much easier time when it comes time to produce either mold or the actual aspheric curved surface that's going to be machined.

Yes there is a downside to using plastic, namely chromatic aberration due to lack of index matching. Other issues are limitations on the bandwidth of transmitted light. Furthermore, most "plastics" used for cell phone lenses have historically been glass-like plastics such as PMMA (polymethylmethyl acrylate) or Zeonex. Most of these types of materials however are extremely sensitive in the UV. The plastic lenses also don't do well with wave front correction or even with just shaping light. To make matters worse, coating plastic is usually problematic as it doesn't do well with higher temperature coatings which tend to aid in not only anti-reflective, but also color correction and glare reduction, i.e. PECVD or other types of enhanced plasma coatings.

Using glass and by glass I mean a crystalline glass, perhaps something closer to a high grade fused silica or if budget permits sapphire or some type of chalcogenide, one gains better modeling control of the index of refraction (abbe number, glass melt, etc..) and also overall optical sensitivities and STOP (structural, thermal and optical) performance parameters. Something else to consider is the coefficient of thermal expansion between glass and metal can more easily be optimized. But all this comes at a cost.

The reason for plastic typically over glass is because of procurement and manufacturing cost. Plastic can have molds machined and have tolerances held rather tightly. Because of this it becomes much easier to pop out lenses. As of late holographic replication and lithographic roll to roll processing has also made making plastic optics a more cost effective option. This is a form of additive manufacturing and actually is probable where things are headed.

Manufacturing "glass optics" especially for aspherical optics are dependent upon first ensuring the substrate is pure and that the glass melts are consistent. In most cases glass must be thermal cycled, typically annealed to ensure and maintain rigidity. Rough machining is then performed essentially hogging out the material, in some cases only yielding between 60 to 70% of useable material. Finally and typically either some type of computer controlled polishing, MRF (magneto rheological finishing) or single point diamond lathing of the crystalline glass is performed. The final finishing steps are extremely costly as a combination of advanced metrology (i.e. MTF, interferometry, point-cloud analysis, non-contact surface analysis, various forms of spectroscopy) has to be performed to ensure that said advanced fabrication has either optimized surfaces for wavefront performance by either optimizing or in most cases eliminating mid-spatial frequencies in the form for micron to nanometer level peaks and valleys, all the while not changing or sacrificing the size and other physical dimensions and also preserving the most sensitive clear aperture (i.e. useable and functional area of the lens). It can be quite a time consuming process.

TLDR; Glass is better than Plastic, but is more costly to manufacture at scale, unless volume and market acceptance can drive the cost down.

what are the applications for non rotationally symmetric lenses?

Thank you!

I think Sony make the sensors only.

Sony only make the sensor, it looks like a bunch of companies manufacture the lenses (sharp, lg infotech, and another company I forgot from my googling 10 seconds ago).

But as the linked thread says, apple designed the lens itself (as it has a patent on it, which is apparently common?)

I am curious what the Samsung lens system looks like given it has real optical zoom

Sony also made the AppleColor 640x480 monitor in 1987 with the release of the Mac II. This was quite likely the most fantastic monitor available in this size in this year. It was rumored to be a regular Sony Computer monitor without an anti-reflective coating so it could be brighter and clearer.

I assume the market is extremely bimodal. Either you are satisfied with a phone camera, or you want to go all out and have all the benefits of a DSLR. There are very, very few people who want a stand-alone camera that offers less than a DSLR.

Point-and-shoot cameras are a dying segment. The target market for this segment is taking casual photos of friends and family. These buyers prioritize convenience and price over performance. Smartphones meet these requirements as good or better than their dedicated counterparts.

> AFAICT, regular cameras with vastly bigger apertures, which should be able to take better photos, in fact are generally much worse until you get into huge bulky DSLRs. Why?

Are you sure about this?

The last point-and-shoot camera I owned I bought around 2010, and other than zoom, my smartphone gives much better results. But then I met someone with a new hobby-level Panasonic point-and-shoot camera and I was flabbergasted at the quality of the photos she gets out of it. I tried taking some pictures with it and compared to my smartphone they all looked they belonged in a magazine. Smartphones have a ton of processing power and sensor quality packed into a small package, but I think it's still hard to outperform a half-decent lens and good quality sensor that are much, much larger, and it seems like point-and-shoot cameras have gotten really good.

Why are the lenses so complex outside of the area that matters? Structural integrity / resistance to dropping?

I think they’re still pretty aspherical, right?

It's possible, but they are sufficiently regular I think some obvious hand optimization math could get you there. I've seen similar in other fields.

Yes. Zemax, CodeV, and other optical design software is fundamentally optimization as a design strategy.

These designs are required due to the package size constraints and the relatively generous lighting conditions.

Not an astronomer, but from my limited knowledge telescopes usually use reflection e.g. mirrors so they don't need to correct for chromatic aberrations introduced by lenses and they also don't use things like optical stabilisation. Refractors could possibly be corrected, but AFAIK they aren't widely used in astronomy and hobby telescopes is such a niche market.

(astronomer here). Some comments below partially answered the question, as modern optical/ir telescopes are based on mirror designs (i.e. they have primary, secondary, and sometimes the tertiary mirror like the Vera Rubin telescope). But the lenses are still heavily used in the camera part of the instrument or in spectrographs. And there is a lot of know-how there, as well as optimization in terms of minimising light losses, aberrations etc. I think the constraints and goals are clearly different from the cameras in iphones, but I'd think the techniques must be pretty similar (although Apple clearly has a ton of money that astronomers don't so it's possible that there are some big innovations there, that I don't know about).

Possibly for refracting telescopes where the light passes through lenses but pretty much all serious telescopes use mirrors to reflect light and therefore couldn't do this. However, I'm sure that you can come up with some wild optics if you are optimizing for size or minimizing distortion.

DSLRs are no match for a Fujifilm GFX100 or a Phase One or the Hubble Space Telescope but none of those users seem to like comparing them as much.

Physics is still the law. As long as your aperture is small, so is the usable amount of light you’re collecting at the top of that lens funnel.

Is this just a software problem? Android has had various night sight modes for several years that make dark views work pretty well?

Well thats definitely not true

Stars: https://www.macrumors.com/2021/10/10/amazing-night-sky-photo...

Street light: https://cdn.vox-cdn.com/thumbor/R51UlZi7g1UPq7b9Be05nWfOP_o=...

Of course you can.

Stars example: https://www.macrumors.com/2021/10/10/amazing-night-sky-photo...

Street light example: https://www.cnet.com/tech/mobile/iphone-13-pro-vs-12-pro-max...

The problem is an artifact of size constraints and plastic lens having far less variation in optical density, that means a lot more work to correctly direct light to avoid various types of distortion.

https://www.pencilofrays.com/lens-design-forms/#mobile has a much more in depth discussion on the how and why if the lens system design (someone referenced this in a reply to the linked thread)

Long gone are my high-school days of grinding and polishing a 20cm mirror in the cellar, carefully using Foucault knife tests to parabolize it. As an undergrad, using Gaussian formulae when matching lenses in eyepieces. In grad school, writing ray-tracing codes to design multi-element lenses. Then, as a postdoc, using Zernike polynomials to estimate optical errors in the hexagonal mirror segments of the Keck 10 meter telescope.

Today's iPhone optics astonish and impress me: A lens built with over a half-dozen aspherical elements. Coordinated imaging with multiple cameras. Wow!