Nikon reveals a lens that captures wide and telephoto images simultaneously
digitalcameraworld.com
digitalcameraworld.com
still no photos, but they say more info will come during CES
However, feed that mess into AI, and it might successfully be able to use it to see both wide and far.
I don't think wavelength filtering will help you here, as you don't control the input at all. Some sort of splitter in the optical chain might, but you'd be halving your imaging photons with all that entails. Or you can have e.g. a telephoto center and a wide fringe. It's an interesting idea.
The processed result would be quite uninteresting to look at: it's a wide-angle photo where the outer portion is much more blurry than the center. Considering the current intended application, the picture probably would probably be pretty mediocre.
This is very specific technology that solves very specific problems (and might one day make its way to smartphones), but not something I'd expect to produce glamor shots right now.
As a photographer who worked at a newspaper in college for a few years, I just think including an image makes sense in an article about a novel lens.
CES should be interesting, this year.
This would be like someone announcing a new RAM innovation -- and people asking what its Cinebench or Geekbench score is.
EDIT: I got pulled into the discussion without reading the article. The lens is for industrial uses.
Without knowing more about the optics, it’s hard to know how much of a role the sensor/ISP play in the innovation, but those are well established and widely capable across both photographic and industrial use cases.
Very curious to eventually learn more about this and whether it might eventually find its way into traditional cameras.
Nikon is an optics company that's also made cameras for a long time, and then very nearly didn't; before the Z mirrorless line took off, the company's future as a camera manufacturer was seriously in doubt. But even a Nikon that had stopped making cameras entirely after the D780 would still be an optics company. There is no serious reason to assume the necessity of some sensor/ISP "special sauce" behind the novel optics announced here to make the system work. And considering where Nikon's sensors actually come from, if there were more than novel optics involved here, I'd expect to see Sony also mentioned in the partnership.
Of course that's not to say photographic art can't be made with commercial or industrial equipment; film hipsters notwithstanding, pictorialism in the digital era has never been more lively. But I would expect this to fall much in that same genre of "check out this wild shit I did with a junkyard/eBay/security system installer buddy find", rather than anything you'd expect to see on the other end of a lens barrel from a Z-mount flange.
If it's for eyeballs it would be nifty to know what kind of image displays both kinds of information at once.
If it's for computers, what is the advantage over two cameras right next to each other? Less hardware? More accurate image recognition? Something else?
Imagine a low-index ball lens in contact with a thin high-index negative lens. That's the idea. I'm sure the real design uses multiple surfaces/elements for each lens, and I'm sure it's hyper-optimized. I'm interested to learn how close my guess is to reality.
Apologies for the complex wording to describe geometry.
I think I understand that the precessor would not be able to read out the sensors at the same time, but time-multiplexed bracketing has been done before, it really should not be too hard or weird to apply that concept to multiple sensors? (some sensors with integrated memory might even be able to do concurrent capture/deferred readout?)
I don't know much more about the computational photography pipeline, but I imagine there might be some tricky bits around focusing across multiple lenses simultaneously, around managing the slight off-axis offset (though that feels more trivial nowadays), and, as you say, around reading from the sensors into memory, but then also how to practically merge or not-merge the various shots. Google already does this with stacked photos that include, say, a computationally blurred/portrait shot alongside the primary sensor capture before that processing was done, so the bones are there for something similar... but to really take advantage of it would likely require some more work.
But this is all by way of saying, this would be really really cool and would open up a lot of potential opportunities.
I’m imagining you open a “photo session” when you open the camera, and all photos taken in that session are grouped together. Later, you can go into each session and some AI or whatever spits out a handful of top edits for you to consider, and you delete the rest.
Use case is for taking photos with children or another animals where you need approx 50 photos to get one where they’re looking at the camera with their eyes open and a smile, then today you need to manually perform some atrocious O(N*K) procedure to get the best K of the N photos.
Pardon my ignorance, but isn't this just an inferior version of after-the-fact photo capture?
On a serious note, what do you really mean by this? I have trouble imagining how that would work.
There are probably a good number of camera apps that support this mode; two I know of are ProCam 8 and Camera M.
But I'm not sure what you mean by "choosing lens/sensor pairs" - do any modern phones even expose that? The samsung version is just "you zoom. Occasionally the field of view jumps awkwardly sideways because it switched lenses."
I want to point the side with the cameras at whatever I want to take a picture of, hit the release button and move on. All that careful framing? That might be enjoyable to do ahead of time if you take pictures of carefully arranged flower bouquets or something like that, but that's very far from many of the use cases of the always-at-hand camera. Give me ultrawide, tele and whatever exists in between at a single press of a button and let me go back to whatever I was doing while the desire to persist a visual situation came up. Memory is cheap, selection is a chore but one that can be done time-shifted. I'm not suggesting to take away your ahead of time framing, I'm just longing for an option to simply read them all (sequentially, if necessary). Perhaps even throw in a readout of the selfie-cam for good measure.
And this I just have a photo by the camera for the camera with the camera is a market taken over by phone now.
But for photography taken by camera, it is specialised these days. You want a picture you choose, you frame, background blur, for the first photo left ey of the subject of the far right corner and the second is on the right eye of … etc. You choose. And that requires choice by the camera man, or person, not the camera.
Or maybe the output is boring. i.e. two different output streams with acceptably sharp image with well controlled distortion.
I'm on mobile, this website has deceptive hitboxes so "dismiss" on the notifications prompt will almost hit the advert on the bottom. Once I finally managed to read the article and hit the bottom of the page, it had a popup which hijacked navigation so I couldn't easily return to HN.
Will record this behavior if asked.
That said... There's not much to say about this? Is it using "meta" lenses printed on silicon? Wish the article would have had any information whatsoever. I suppose we'll learn more at CES. :P
The other option is concentric optics with a pick out mirror for the central light path. Bit harder to make but you get more flexibility re: how much of your light collecting area gets split to which sensor.
Both images end up superimposed on the sensor, and there is probably a lot of distortion too, but for AI that might not be an issue.
Various designs for microlens arrays do similar things - thousands of of 0.001 megapixel images from slightly different angles are fairly useless for most human uses, but to AI it could be a very powerful way to get depth info, cut the camera thickness by 10x, and have infinite depth of focus.
I did a bunch of manual creation of light-field photos over the years.[1] To get interesting compositions, you need an effective lens diameter of about 30 cm in diameter or more. To get super-resolution good enough for zoom, you're going to probably need something that size.
[1] https://www.flickr.com/photos/---mike---/albums/721777202979...
Like, why even mention them?
If it produces an EXR with clearly seperate images with different lenses, fine. Like a 3D EXR with left and right.
If you plot pixels per degree over the field of view of a fisheye lens you will see that vastly more pixels are dedicated to the center "eye". And also the field of view is large. Which is what this novel lens claims to also do.
There are companies that make stereo lenses, capturing two images side-by-side on a single sensor, for people who want to take 3D photos on their interchangeable-lens cameras. And there are "anamorphic" lenses that squeeze things horizontally but not vertically - in digital terms, producing non-square pixels. Very popular in films in the 70s and 80s. And when it comes to corrective glasses, bifocal and varifocal/progressive lenses are another common type of lens providing variable optical properties.
Self-driving cars need to deal with both "stopped at a crosswalk, are there pedestrians?" (which needs a wide view) and "driving at 70mph, stopping distance about 300 feet, what's that thing 300 feet away?" (which needs a zoomed in view)
If you look at https://www.pexels.com/photo/city-street-in-fisheye-16209078... for example - it's wide (which is good) but the details at 300 feet ahead aren't winning any prizes. Far more pixels are wasted on useless sky than are used on the road ahead.
Fortunately you are wrong.
Maybe their lens is variable focal length, providing more magnification in the center at a presumed cost of clarity.
(Differing projections/distortions such as fisheye are likewise exactly equivalent to mathematical transforms for the same reason.)
You can see that this is true when using a (physical) zoom lens. When zooming, there is no change in the projected image of any sort other than it simply grows larger.
The effect you are referring to is complementary to parallax and is likewise due exactly to physical proximity to or distance from the subject being photographed. (Telephoto lenses require the subject to be further away to remain in the frame; in doing so, they move relatively closer to the background, thus longitudinally compressing the scene.)
The thing is, distance and focal length aren't independent. People don't usually shoot long lenses in close quarters, and don't shoot distant subjects with wide angle lenses, we just tend to fill our frames with the subject. That means the compression effect is technically not related to focal length, but in practice ends up showing up more when using longer lenses.
Generally, I think the answer is yes. But the more complicated answer is that every lens design has its own flavor of distortion. An image from a lens optimized for telephoto shots is going to have slightly different characteristics than a cropped image from a lens optimized for wide shots.
The compressed effect I think you are referring to is likely attributable to perspective, most noticeable when the movie does one of those zoom shots where they keep the subject at the same relative size in the frame while moving the camera in or out. Like on this shot: https://youtu.be/in_mAvHu9E4?t=19
Either way, I'd expect it to need significant amount of processing to get anything useful.
What would be the optimal sensor geometry for such a lens? The distortion would be crazy, wouldn't it? Nowhere near a rectilinear projection.
Most lenses are "distortion corrected" because they assume they will be displayed on a flat surface. A little explanation for those not familiar: When you take a picture of a brick wall with your camera parallel to the wall, notice that the bricks on the edge of the photo are the same size as the ones in the center, even though the edge bricks are further away from the camera. This means more pixels are allocated per degree of view near the edge of the field-of-view than at the center.
An "uncorrected" lens is basically what we would call a "fish-eye" lens. Here (ideally) the same number of pixels are in a one-degree circle in the center of the field-of-view as are in a one-degree circle near the edge.
I don't think they would crow about just using an "uncorrected" lens either, so I'm going to guess that this is a "reverse-corrected" lens system where a one-degree circle in the center gets more pixels than it would at the edge. This would be the obvious approach if they want a good center crop but want to capture all the periphery as well.
Given the automotive context of this product, I would expect the goal was to maximize the resolution for center FOV (more clearly resolve objects further down the road) while simultaneously maximizing the overall FOV angle (see closer objects in peripheral vision).
In practice, the raw image probably looks like the old Photoshop "bulge" filter, and then somewhere in the image pipeline, it will get reprojected into a regular image that is quite blurry at the edges and becomes increasingly high-res toward the center.
(Another way of looking at this would be that this is an optical adaptation to the uniform, cartesian nature of image sensors, allowing a "foveated" image without a gradation in sensor pixel density.)
I don't have a good sense for what the R&D required to spin up a new bespoke sensor is, but I think it's sort of high - I assume there's a reason Nikon seem to source most of their sensors from Sony. Assuming you get to use a custom sensor for your camera, you also lose a bit of sensitivity at the "fovea."
Also, Nikon's own press release also refer specifically to an "optical lens system with both telephoto and wide-angle lens functions," which leads me to believe this isn't an innovation at the sensor level.
https://www.escherinhetpaleis.nl/escher-today/balcony/?lang=...
Even high-end full frame lenses + sensors with a fixed focal length struggle to reach 200MP of detail. (60MP Sony A7RV with pixel shift can take pictures with 240MP). No way this weird monstrosity can get anywhere near 200MP in a moving vehicle.
The depth of field is determined by the focus distance and the aperture of the lens. Both remain unchanged.
Note that 35mm F/2.0 is the same aperture as 70mm F/4.0. Both lenses have an aperture of 17.5mm. (35/2.0 == 70/4.0)
You can easily verify this with your favorite zoom lens. If you have an 24-70 F/2.8 available to you, you can verify by taking 2 pictures; one at 35mm F/2.8 and one at 70mm F/5.6. Crop the 35mm one to 25% area (half the width, half the height). Render both images to the same size (print, fill screen, whatever) and see for yourself.
this is an assumption that goes against the concept of "f-number" so if one does it, they should not expect to get to anything sensible.