Sony's new curved image sensors could shake up the whole camera industry | Hacker News Reader

Sony's new curved image sensors could shake up the whole camera industry

digitalcameraworld.com

177 pointshhs114 comments

It is looking like Nuclear Fusion or Battery Breakthrough. Consider how little media hype it is getting and no actual production time line, which means the absolute earliest commercial usage will likely be 4 - 5 years, takes another 2 - 3 years to iron out all the edge cases, we are looking at 2030 before it becomes mainstream.

I dont follow lens and sensor industry at all so do correct me if I am wrong.

Previous Discussion

https://news.ycombinator.com/item?id=25113755

Submission from 7 years ago.

https://news.ycombinator.com/item?id=8006235

Is there a term for that? Like vaporware but not the same, something that could possibly exist some day due to some lab work but is being hyped by the media as some breakthrough.

You're actually describing vaporware. Vaporware certain to not materialize is downright lying/fraud I think.

https://www.gartner.com/en/research/methodologies/gartner-hy...

I'm not sure but there's always been a running joke about nuclear fusion always being 20 years away.

I think nuclear fusion power plants are being designed on mainstream desktop Linux workstations right as we speak! :-D

Nuclear fusion seems much further off than either of the other two. I'd be surprised if it becomes mainstream before 2100 if ever, whereas sometime before or around 2030 seems reasonable for the solid state batteries and curved camera sensors.

Thanks for that clarification, nuclear fusion looked more like lasers and large proof-of-concept demonstrations in national laboratories the last time I checked. Sensors and batteries may have MUCH more contracted timelines.

I think OLED followed that timeline (actually, more than only one decade), and we have OLED everywhere today. It could take a while, but don't discount its potential to come to mass market. And FWIW, nuclear fusion and battery breakthroughs are also finally starting to see some new progress, though that also took more than one decade.

I recall watching a show about a university investigating a material from marshes that may someday be used for TV screens.

The writer obviously does not understand the benefit of curved sensors. They will make the lenses cheaper and smaller - they will not change the way photographs look.

Making the sensor smaller and cheaper also enables larger sensors at the equivalent cost and form factor of a current sensor though, which would change photos (if you think higher output resolution is a change to an image). The benefit doesn't have to be taken as making things cheaper.

Full-frame at APS-C prices would be great. It would lose one of the perks of APS-C: crop factor. Getting 450mm out of a 70-300mm lens is nice.

edit: since it's apparently not clear and I'm getting downvoted for sharing how this benefits a harmless personal preference, I will emphasize that I'm speaking to my needs. The 70-300 lens is what I use, not some abstract construct meant to represent all purposes.

The crop factor is generally a perk for people who shoot at the ranges I shoot at even if it comes with compromises for other needs.

this is essentially zoom though as you're not using the full output of the lens

That's okay. I usually shoot things that are further away. Other people will have different needs. Only speaking for myself.

Isn't this usually thought of as a disadvantage? E.g. You have to buy more glass ($$) to get smaller mm equivalents?

I am only speaking for my use cases. Nikon crop sensors can use DX lenses, and those are always cheaper for more or less the same reach and features.

Crop factor is only an advantage for APS-C cameras where the pixel density is higher than the comparable FF camera. That's not always true, eg I have a Sony a7Riv, which has 26MP in crop mode (more than APS-C cameras).

The "croppability" of the image is not really a function of the sensor size, it's a function of pixel density. APS-C cameras used to have higher pixel density than FF cameras, but that hasn't been universally true recently.

Or where you are willing to sacrifice pixels to get the longer length.

Lens size is the limiting factor for smartphone cameras, so the photos will look better.

I mean you're not wrong, but the author here understands that the main application is for compact-sized phone camera modules, which in their case the lens et al. are nearing or already in their physical limits.

> They will make the lenses cheaper and smaller - they will not change the way photographs look.

This article notes that curved sensors will enable "greater aperture and reduced light fall-off at the edge of the photo", and other articles on curved sensors suggest that they require fewer optical elements (which I assume has associated quality benefits) and will reduce distortion and vignetting when shooting at wide angles. Are these not notable improvements beyond cheaper/smaller?

Very clear the writer hasn't taken a course on optics:

"Rather than follow the curve of our eyes, the sensors in conventional digital cameras are flat. This results in an unnatural curvature in the image, and so the accompanying lenses have to be made in a way that corrects this distortion."

.... The aberration that curved image sensors helps mitigate is spherical aberration. Not distortion ....

Better, cheaper, and/or smaller lenses if the design is made specifically for the curved sensor.

The Agfa Clack, an instamatic-type camera made in the 1950s, had a slightly curved film plane to correct for the cheap plastic lens.

I was gonna say this reminds me of the curved film planes on cheap plastic cameras including the back of many disposable cameras. Almost as if it's good to make the focal point the same distance away from the lens in all places.

Notably, this is for mobile phone sensors, which are highly "physics constrained" and need every trick in the book to achieve decent image quality.

Indeed, yet you see people doing interesting gimmicks to take a picture with their tablet instead of carrying around a proper camera.

They say the best camera is the one you have on you at the time. This holds true with my experience; even with a dedicated camera I have many more photos taken with my smartphone.

I agree, and the compact camera diminishing market shows that.

Yep, my travel photos are split between DSLR photos and phone photos, the phone photos are usually way more interesting because they were of things I was otherwise never going to capture.

mine are split between phone photos and tough camera photos, which are of worse quality than the phone ones but...

> I was otherwise never going to capture

;)

Conversely I have a good DSLR with very good glass attached, as well as M4/3 pocket camera, and two phones. After the trip I dump everything into one Lightroom album and go through picking out the keepers without looking at the specific camera model used. My typical result is 80% DSLR, 10% pocket camera, 10% phones.

What do you think is the biggest "keeper" differentiator for you, between the three, that favors the DSLR? For me, it used to be dynamic range, but now I basically can't making my (older) DSLR pictures look as good as my iPhone pictures.

The resolution and detail on a “real” camera is killer for me. I can save photos with the crop much more easily. Also, portrait mode is good, but nothing beats real bokeh.

The reason I bought my A7II is so that I can: * Get more light in with massive full frame lenses (and hence less noise) * Ability to use lens hoods, xenon flashes, easier to use tripods * Have more lens options available (even though the prices make me cry) that use the same (main) sensor (as opposed to phones with multiple lenses, telephoto etc tend to use low res/smaller sensors) * (the biggest point) so I can take a photo, have the RAW and know that the only thing applied to it is the colour profile; no AI photo magic, no Apple saturation boosting, etc

When I take a photo with my A7II I know what I have on the card is the photo _I_ took. With a phone who tf knows what the image processing chip has done to it to make it "look good".

I found a nice compromise in Fuji's crop system. The cameras are about as small as can be while still retaining reasonable handling (having tons of mechanical knobs helps). My experience was that outside of some extreme special cases I don't want to make use of the large apertures enabled by full frame, the depth of field is just too small. And if you don't need that light gathering capability you can get smaller lenses. It's pretty nice being able to fit camera with a normal zoom, tele-zoom, wide pancake and normal prime into a 3L sling bag.

Regarding processing vs. RAW, all images are processed, there really is no such thing as a "true" photo. If the on device processed photo looks good then life is so much easier. If you can't trust the device processing or want to tweak the last millimetre out of the photo, even Apple devices can give you the RAW sensor data these days.

I'm not them, but I have a similar process. I've got a mirrorless instead of DSLR, and a supertelephoto zoom lens (Sony a7R4 with 200-600mm f/5.6-6.3 lens). I've also got a superzoom (Tamron 28-200mm f/2.8-5.6) lens for closeup and occasional landscape images.

I'm mostly interested in taking pictures of birds, including in flight. So long focal length is the biggest benefit. High resolution (allowing for deep crops while still getting a decent sized output) is another benefit. Good tracking auto-focus is another.

I use my phone mostly if there's a pretty landscape or flower and I don't want to switch lenses. I'm also willing to use my phone in some situations I wouldn't risk the camera & lens, eg while kayaking (in a waterproof case with lanyard). And of course the phone is lighter, so in everyday situations I'll have it, if I want to take a picture.

There're compact cameras with decent quality and not overly big or heavy. I use Canon G9X Mark II. It produces photos of much better quality than the best smartphone camera I had (without software tricks that could be applied to Canon photos afterwards anyway), and it is small enough to carry it with me all the time in the pocket.

the tricks with high dynamic range (software combine two shots), night photos (take loads of photos and pick the sharpest bits vs tripod) are way more work to do after the fact

And then you want to apply a snazzy filter and instantly share your photo to all your friends via whatsapp or instagram.

When I have a feeling that I would like to share the picture with my friends instantly, I also take it on my phone.

Also, Canon G9X has bluetooth and wifi interfaces and phone app that allows quick downloading and sharing of photos (although it is indeed slower and more cumbersome than sharing directly from the phone camera app).

I've got a Powershot and the problem I find, compared with my phone, is the time it takes from "oh there's something" to actually being able to take a photo - for my phone, it's maybe 2-3s (if the Camera app isn't already open; more like 1-2s if it is.) For the Powershot, it's always 10s+.

Back in the olden days, cheap cameras had no latency at all. Just press the button and it takes the picture instantly. I've often tried to find something like that but things keep getting worse. Iphone tries to help with a camera on the lock screen but there's still a lot of tapping and swiping and waiting.

I find my mirrorless Sony 6700 to be pretty rapid, it's pretty much ready in the time it takes me to flip the switch and move my head to the viewfinder.

Get a lot of photos of the inside of my lens cap that way.

IMO, phones should have a hardware shutter release button. Press it at any time to take a photo immediately, regardless of the software context. It's bizarre to me that our $1000 phones are still subject to the micro-optimization that leads to elimination of buttons as a primary design goal for very cheap devices.

https://www.dpreview.com/forums/thread/4546583

It's frustrating to me that these are the exception. That may not be my absolute top priority in choosing a phone, and if it isn't, chances are I won't get it.

Vote with your wallet then.

> Press it at any time to take a photo immediately,

The internet three days after Apple do this: "APPLE PHONES CONSTANTLY VIDEOING SURROUNDINGS, 1984 IS HERE"

If apple was scanning the video for potential evil and contacting the FBI when evil was spotted, then yes people would say "1984 IS HERE".

Camera modules and image processing take a lot of power.

Ok, maybe "immediately" is too ambitious, the camera doesn't need to be powered on all the time. I just want a hardware button that gets you to camera-is-ready ASAP.

> Back in the olden days, cheap cameras had no latency at all.

Some of 'em, yeah. Others where the lens has to come out of the body before you can take a photo, somewhat different.

(And yes, I suppose you can leave them powered on to keep the lens out of the body but then you're risking scratches or mechanism damage when you put it away and they also aggressively auto-power off.)

This reminds me of my wonderful 2013 Google Glass: 1) Say "OK Google, take a picture" 2) 2 seconds later, you see the picture it's taken in the little window

This is confusing to me. One of the main reasons I have a DSLR is because the time from "I want a photo" to pressing the shutter release button is still the shortest with a DSLR. Tactile buttons, very fast power-on, very fast shutter release response time.

> One of the main reasons I have a DSLR

I can't fit my DSLRs into my pockets though which means they don't get carried everywhere like a phone or compact would.

Opposite experience for me. I can turn on my DSLR, zoom, focus on the subject, and get a shot in under two seconds, with no noticeable latency between pressing the shutter and taking the photo, and without having to take my eyes off the subject. Subsequent photos a half second or less.

(I used to own a Canon G11. It was not as fast as a DSLR, but I don't recall it being nearly as slow as you describe.)

Meanwhile on Android, I'm lucky if the stock camera app loads in that time. Plus another three seconds to muck with the focus circle, and another two seconds of latency while it actually takes the photo. The whole time I'll be starting at the screen trying to find the tiny circles for my fingers to tap. If I managed not to actually miss the moment (unlikely), I later get to go back and crop the photo, because pinch zoom while shooting is too slow to even consider using while shooting.

I have missed a lot of good shots when all I had was my cell phone.

(Yes, I could get an iPhone and it will probably perform better, and buy some contraption to give it physical buttons.)

> I can turn on my DSLR, zoom, focus on the subject, and get a shot in under two seconds

To be fair, I can probably do that on my DSLRs as well if they're left in standby mode -but- there's a certain amount of weight and bulkiness inherent here that moves it out of "why is a phone better than a compact" territory because I damn sure can't fit any of my DSLRs into my pockets.

I don't know if DSLRs changed while I wasn't looking, but my old Nikon D5100 turns on and takes photos without delay, i.e. as fast as my fingers can actuate the levers. My phone takes 5s to wake up, 5-15s for the camera app to wake up. It is actually kind of a travesty how slow it works (Android on Nexus 6P) given how modern the hardware and software is. Clearly you could run a lean BeOS style system on it that would rival the DSLR.

It's slightly slower with the mirrorless systems because you have to wait until the viewfinder is ready before looking through it. It's fast enough, but the viewfinder is black while the camera is off.

It certainly must be due to the mirrorless switch. The older DSLR is just good enough that I never ended up trying out any of the newer systems.

I just tested my Sony a7R4 mirrorless camera with a stopwatch. It takes about 0.6s for the viewfinder to be ready. If I flick the switch while raising the camera to my eye, it's up just as I look through.

I just sang happy birthday at what I find a normal pace in 15 seconds. If you can't open your phone and camera app faster than you can sing that song then maybe you should look into upgrading your phone. My iPhone from the screen being off to the camera app finished with taking the first picture is easily under 2 seconds, probably about half that.

My DSLR is about the same

I don't disagree. Doesn't make me any less grumpy about the state of things in app design and lack of respect for low latency in UI though :-)

"There was once a dream that was Rome, you could only whisper it. Anything more than a whisper and it would vanish." Marcus Aurelius, ‘The Gladiator’.

What Android phone do you have? I've just tested on an S21 and the camera app loaded in less than 1 second. Felt more like 0.25 or 0.5 seconds at worst.

Don't forget to count time activating the phone itself. Admittedly I have a cruddy phone (Moto G4), but the phone app itself takes about 1.5 seconds to start, on top of 0.5-several seconds to come out of sleep mode (sometimes it takes 5-10 seconds). And if the lock screen is up, opening the camera takes longer due to the weird swiping motion needed which always takes me a few tries.

Contrast with my DSLR, the on switch is colocated with the shutter. I can turn it on and get a photo in 1.5 seconds.

I just leave my D5600 on all the time and let it go into standby. It can sit for months like that and still take hundreds of photos. It wakes up and focuses with a single half press.

Double tapping the power button on my S21 brings up the camera. Tried it now and it's almost instant.

With how fast mobile SoCs are becoming, I think this will be even faster.

There certainly are genres of photography where you absolutely have to take a picture of a fleeting moment in under two seconds, but I don't think it is a common scenario for most people. The things people photograph the most are people (incl. themselves), and landscapes/objects/things. And I say this as someone who recently dumped $6.5k on a camera for wildlife. I obviously do see the advantages of dedicated cameras, but for casual users I don't think they'd need more than a phone camera. $.02

Small kids generate tons of such fleeting moments and also need good autofocus and fast shutter speeds.

Unfortunately the compact camera market is dying. We probably won't see many new models from Canon and their competitors. It's a classic case of disruptive innovation, where cell phone cameras are still technically worse than dedicated compact cameras, but still good enough for most people. Olympus just sold off their camera division.

Smartphone cameras are getting better at an incredible pace, though. And they're probably a few generations (5, maybe 10 years?) of hardware and software away from being indistinguishable from pro photos for casual users, it seems.

They are still only collecting a limited number of photons. That's a fundamental limit to how good a camera can be, and size helps massively. They are also diffraction-limited, meaning that at the size of the lens they cannot be any more sharp. These are hard physical limits that we're getting really close to. A DSLR with a much bigger lens and much bigger sensor is always going to produce better pictures.

True, however they are making use of machine learning to add the missing pixels.

How much of a real photo that actually is, is debatable, however many users will probably be happy with the result.

Yeah, but as Apple used to say: "it's not about speeds and feeds". At some point for the average human eye the "speeds and feeds" provided by smartphones, when coupled with smart combined lens usage and machine learning, will be enough.

Pixel peeping is really rare and photo accuracy doesn't matter as much for the average person seeing a photo on a small device. They're not going to print that photo on glossy paper for a posh magazine or put it on a huge billboard.

> the compact camera market is dying. > Olympus just sold off their camera division.

And someone _bought_ that camera division. With the intent to continue designing, manufacturing and selling cameras under the Olympus brand. Sure, Olympus wasn’t doing great, but someone paid money because they thought they could run the business better. And their consumer business is in selling compact cameras, right? There appears to be at least some faith in the market.

> We probably won't see many new models from Canon

Personally I sure hope so. Their less-than-full-frame offering has been pretty lackluster afaict, other companies are doing the job better. We’ve lately seen some surprisingly kickass tech innovation in the full frame market from Canon, though. I hope they continue to focus their efforts.

> Notably, this is for mobile phone sensors, which are highly "physics constrained" and need every trick in the book to achieve decent image quality.

How about cameras used in endoscopy?

Or drones.

Drone cameras have more than 7mm to work with.

Not insect-sized drones

It's for a single axis curve, where the sensor is thinned and then bent over a form (presumably along the long axis), rather than a fully dished image plane.

If the goal is reduced complexity / thinner optics, I guess this can make the design problem easier even if it doesn't solve it completely? Although I have to assume that you'd end up with a host of new distortion/aberration problems.

I'm also assuming that this is only being considered for smartphones and not for ILCs? For ILCs, most lenses can be adapted by knowing one number -- the flange focal distance. Needing to also deal with image plane curvature would throw a pretty big wrench into that...

> Needing to also deal with image plane curvature would throw a pretty big wrench into that...

While opening a host of opportunities for selling lenses.

Check out the Schmidt camera to see what little glass can give you great optical performance if you can live with a curved image plane. The Baker-Nunn camera puts this to the extreme: 500mm, f/1, 30 degree FOV.

Even a simple Schmidt–Väisälä in 400mm f/2 can cover a 4000px wide sensor with 5.5 micrometer pixel size perfectly (even 3.5 micrometer pixels would still be useful, especially if color).

> It's for a single axis curve

This is required if you're to display the images on a square display, without distortion, correct?

Not really. As long as you've got a grid of pixels, it doesn't much matter what their shape is. Of course having a regular pixel sizing would end up with distorted images. Correcting for lens distortion isn't exactly a new trick, but if you want to be really fancy about it you could probably make non-uniform pixels too, correcting for it in hardware.

I think most photo companies (and even MS) have patented curved image sensors at this point. Everybody has been saying that it's the next big thing, but it's probably hard to produce and I'm not an optics engineer to actually chime in on how much improvement it can have.

The article touches on that and seems to be saying the significance here is that it’s a patent for the production process of the curved sensors, not just a patent for the apparatus itself.

Cool, now just wire that image directly to the brain, so that the effect isn't ruined by going through an actual second eye.

Call me a party pooper, but I believe the greatest achievement in optics is that lens type which crisply maps straight lines to straight lines. When you then view the resulting image from the proper distance, you see what the eye would have seen at the original scene. If there is any curvature apparent in the straight lines, it's coming from you in the same way; you're getting your "curved sensor industry shake-up fix" right there.

I don't think the purpose is to make a different shaped image, but to get the same image with smaller optics. If you wanted it distorted, you could just post-process the image from a flat sensor.

I'm sceptical.

First time I've heard about curved image sensors ~15 years ago. So until this thing ships in an actual product, I'll put in into the same category where I put revolutionary batteries with 20x capacity (promised since 1990) and other too good to be true things.

the power density of AA batteries may not have changed much because AA batteries need to be disposable in ordinary trash

but if you have used a notebook in the 90s and one now, how can you say that capacity hasn't improved?

If the author is raving about vertebrate eyes, wait until they discover the squid.

There are at least three domains for solving distortion: Lenses, sensors, and software. This could be a significant breakthrough IF the economics of making curved sensors is better than the economics of making better lenses or using a higher resolution sensor to compensate for throwing away part of an image in correcting distortion in software.

There is a significant chance this will end up like the Foveon sensor: Better on paper but not better enough to overcome "let's throw more pixels and software at it."

not an expert in optics, but I thought the limitation of curved sensors in achieving the hypothetical reduction in lens complexity is every focal length would require a different curve radius? So you'd have one focal length that would be significantly simplified because the curve of the sensor would match the curve of the focal "plane" and any other lens you attach would have to become equally or more complex to match its curve to the curve of the sensor? I know for cell phones this wouldn't be an issue, because they have a single fixed lens, but for all the cameras with interchangeable optics, does flat makes more sense?

EDIT: after a bit of thought, if I'm correct above this would also benefit optics in many industrial applications. My brain is usually rooted in cameras people take pictures with, but there are optics and sensors in a lot of other applications. And of course saying "just cell phone cameras" is funny because they are by far the most popular cameras in the world, just that I never use them so they are a bit of blind spot for me.

I remembered that I saw news of another company patiented transformable lense on hackernews. Probably that combine to this can reproduce human eye completely (high focal range and compact at same time)?

What I'm interested to see is when/if a camera like the Lytro will be put into a smartphone.

Does it work if the pixels are much smaller than the diffraction limit?

If it requires me to buy new lenses then no thanks.

I think the huge potential is lenses may become cheaper, given they don't need layers of precisely curved glass anymore.

That said, if only Sony is making these, they'll just keep the prices the same to rake in more profits.

> That said, if only Sony is making these, they'll just keep the prices the same to rake in more profits.

You seem to be suggesting that Sony is price gouging or something when in fact they are probably the most innovative camera company and their GM lenses are fairly and competitively priced against modern lenses for Canon RF mount and Nikon Z mount.

If only sony is making these, a lot of other cameras will see them as well since sony is the sensor manufacturer for say, nikon. That' doesn't mean prices will go down, however.

> I think the huge potential is lenses may become cheaper, given they don't need layers of precisely curved glass anymore.

They'll still need precise optics to prevent distortion, chromatoc aberration and coma. And to maintain sharpness. The only advantage the curved sensor brings is that the lens doesn't need a final group to force the image onto a planar surface.

I would love to have a lighter version of the 70-200 though...

So no innovation should ever be made again?

This patent is aimed at smartphones

> 108MP sensors becoming increasingly common

Do millions of tiny binned pixels provide better image quality compared to much larger pixels with the same sensor area? How do they compare in low light?

More pixels in low light could allow you to do more intelligent processing, maybe even in a way that allows for longer exposures or multiple exposures to do the job of gathering more light (without incurring as much of a fidelity penalty as usual from motion blur.)

More pixels provide more details, but not always more quality, as poor lens quality is also "enhanced". This is useful for large prints which require a high dpi. In low light, less pixels in a greater sensor area perform better. Dense pixels result in more iso noise

Diffraction blurs the image even for 10MP, given the small lens/sensor. The other 98 MP are used to remove noise, especially at low light

I assume this is similar to delta-sigma ADC/DAC, where you shift all the noise to a higher spectrum, so you can easily cut it all off?

I dunno, I took optics 15 years ago and only one signals class :P

I'd just average the lowest (N-1) pixels. N-1 because shot noise is a very bright pixel - if I have a bright pixel not including it is best for the avg. If the pixel is not surging, not including it doesn't affect my average much.

Given the same sensor area, a dramatic increase in pixels will reduce low-light sensitivity. The space between pixels does not collect light, so the more spaces between, the less light collected overall.

Think of a tic-tac-toe grid - it's mostly white space with a few black lines. Expand that out to a 1000x1000 grid in the same space, and there is a lot less white space and a lot more black lines.

We have been using gapless microlenses on imaging sensors for quite a while now. Also, when the pixels shrink, tech advancements in reducing downstream read-noise also mean that very little ISO performance is sacrificed (if any). Certainly, there is a lot more nuance and tech to discuss, but those are the broad strokes.

https://www.photonstophotos.net/Charts/PDR.htm#Canon%20EOS%2...

https://www.photonstophotos.net/Charts/RN_e.htm#Canon%20EOS%...

Seems a bit optimistic to simulate one of the properties the eye has (by necessity) and hope that this somehow improves image quality.

Reply on news.ycombinator.com