Apple's camera design choices from the iPhone 6S Plus to the 12 Pro Max [pdf]
systemplus.fr
systemplus.fr
Makes me wish that these big tech companies would release the full uncensored data for their obsolete products. How nice would it be if Apple shared a huge archive of technical info on the original iphone. Surely there is nothing left on that device that is secret or of any use outside of curiosity.
There’s almost zero upside for companies to go through all of the effort of preparing and releasing that information, however trivial it may seem from the outside.
They can be had for $10 a pop on Alibaba now, and they're much nicer than similarly priced webcams let alone LIDARs.
Seems like a part that's ripe for a raspberry pi interface board.
Actually, since that project above links to a iPhone 4 screen as an example, I'd perhaps err into probing for MIPI signals on an iPhone 6S camera first, perhaps using sth like https://hackaday.com/2018/11/29/mipi-csi-2-implementation-in...?
This post is fascinating, btw: https://www.circuitvalley.com/2020/01/spi-mipi-bridge-fpga-v...
AMS provides Apple with optical sensors for 3D facial recognition features on its newest smartphones
“We see a risk that Apple moves to dual sourcing for the face ID – which currently is single sourced from AMS - in order not to rely on deliveries from just one supplier and also in order to have a favorable pricing power,” said Hauck & Aufhaeuser analysts in a note to clients.
Not to mention the “neural engine” that’s used for depth processing and segmentation.
Optics (glass) does not play much role in phone camera cost or product development; but the cost (effort) is a factor of sensor, CPUs, NPUs, DSPs (hardware) and then investment in writing image processing software.
Not everyone wants artificially manipulated photos that you can instantly upload to instagram without any post work. But, you do need good optics and sensor to capture good photos - the rest is post manipulation. Do you think RED or ARRI cameras cost $50k+ for their built in software? Nope, they don't actually have any of the "neural net" hype you've bought into from Apple's marketing. It's all done in post. Practically nothing you see in the movies or TV or Netflix or other professional broadcast is done on Apple toy cameras using their state of the art "neural engine" software. I haven't read about or seen any, but I reserve the right to be wrong so I said "practically."
Besides, you got to start somewhere and the camera interface would come before the splashy software built on top of it.
Just give me the RAW/flat log files (that contain the most information and dynamic range) and I will do the rest with a real editing program that cost almost as much as your phone itself, and look way better.
I’m not deeply familiar with high-end professional cameras, but low-mid pro cameras sort of slept for a long time while phone cameras had to play around the trivial upgrade route of bigger sensors and go the harder route to more intelligent sensors. There is a really interesting article about it that I may try to dig out, but there are such great tricks in modern phones like using the not that great lens stabilization to an advantage where minor movements of our hand will render the same physical position to several adjacent “colored” sensor positions and a clever NN can use this additional information to increase the lost sensors size inherent in colored picture taking. Also, HDR photos simply need intelligence.
All in all, in certain (rare) situations a newer iphone may very well shoot a much better photo than a DSLR camera no matter the comparatively tiny sensor size. Also, just think about the way the black hole was photographed - it also didn’t use an Earth-sized sensor, but the effect was similar if it have used one.
Short of a 3CCD prism setup, most cameras use Bayer (or other) filters that capture a close but false image that then is reprocessed back to an approximation of the scene.
When people talk about smartphones having better processing, it just means on top of demosaicing, the software might be applying additional algorithms.
P.S. RED cameras don’t use 3CCD yet still cost $50k+ implying artificial manipulation is the way to go due to cost and physical limitations.
Because I tend to be sceptical of those articles. They start with the premise of "everything is worth as much as the sum of its component parts", which is already hit on their credibility (if they did the same, recursively, they'd notice that nothing is worth anything). Then, they guess some random values for Apple's purchasing costs.
Their youtube chain is really interesting, you should check it out: https://www.youtube.com/channel/UCj--iMtToRO_cGG_fpmP5XQ
It seems like Chinese companies figured out a while back that they need to make their software at least seem good if they want to capture the global market. Obviously the price is an even bigger aspect of the equation.
Then i activated the uncompressed pro format and got much better results (with images weighting a ton more of course)
i still have to get back to the se2 and see if i have the same problems but it was the first time i was completely unimpressed by the camera when upgrading my phone.
If it’s taking blurry photos still, it probably is defective and needs to be returned.
[1] https://piunikaweb.com/2020/12/09/iphone-12-pro-camera-focus...
Moving to the pro format solved the issue, so my guess is that it's more a problem with the compression / post processing than with the auto-focusing part (although i must say i'm not really sure what constitutes post-processing anymore with modern camera)
The original images:
https://drive.google.com/drive/folders/1-2NQ0meQ-PgNvgObWmPT...
A cropped comparison saved as a high quality JPEG, for easier viewing:
https://drive.google.com/file/d/1VEMVPGDE7ck-0du9QYxydM1a3P8...
I don't think the compression is the issue here, by the way. It's just that when you don't shoot RAW you are getting the default denoising+sharpening.
But why ?? isn't a camera supposed to give you "crisp" pictures instead of artistic / smeared renditions of reality ?
I thought the blame could be put on AI-based post-processing going out of control, but your comment seems to imply it's made on purpose. I really don't get the point.
If you look at the images at 1:1, they sometimes look like watercolours with regards to edges.
All phone cameras have tiny apertures and tiny pixels (relatively speaking to their DSLR and medium format counterparts). One can’t use them for any kind of serious photography.
My iPhone 11 takes photos that are 4000 x 3000 RGB pixels. That's 12 million picture elements, or 36 million sub-pixels. It's rated as 12 MP.
Sorry if the writeup isn't that specific, I mostly work with monochrome CMOS cameras.
https://en.wikipedia.org/wiki/Demosaicing https://en.wikipedia.org/wiki/Bayer_filter
edit: I should also state that I don't know anything about iPhone cameras. It's quite possible, but not typical that they have a 36 MP sensor producing 12 MP images.
edit 2: I read that the iphone 12 has 1.7 um pixels. A 36 MP 4:3 sensor with 1.7 um pixels would be 8.3 mm. A 12 MP 4:3 sensor would be just 6.8 mm wide.
The promoted size is the number of photosites on the sensor, but each photosite is grayscale. Look at any RAW camera format or the datasheet of a sensor (e.g. one of popular Sony sensors). All of that applies to Bayer sensor and not Foveon, but Foveon is not particularly popular by any measure.
The size of the sensor and the smarts behind it matter a lot more
For example, the f31d was great in its time and showed that megapixels weren't the only measure to look for.
And the lenses.
My backup DSLR shooting at 6MP with $100 of glass on the front looks infinitely better than the photos out of my iPhone 12 Pro.
I'm thinking if it were a good idea, then Canon, Sony, and Nikon would all be doing that.
I bought an iPhone 12 Pro pretty much entirely as a camera to use to capture a bunch of photos of my kid because (1) I don't have my DSLR within arm's reach in the house all the time and (2) our house isn't very well lit. I have zero complaints about it. I probably take photos with my iPhone versus my actual photo gear at least 20:1, if not more.
The photos look great on a phone screen.
But even just printing a 4x6 photo from the iPhone and a 15 year old DSLR and putting them side by side, the difference is immediately obvious. The iPhone photos are disgustingly oversharpened while somehow being weirdly smoothed, but in all the wrong spots.
I grabbed a couple of my own photos taken on the same day in the same place under largely the same lighting conditions (it's never exact) and zoomed/cropped them to help demonstrate and for privacy. Neither have been adjusted at all (I did try and improve the iPhone photo best I could, but really managed no substantial looking improvement so posted it as-shot), so focus more on the definition in the hair: https://imgur.com/a/7bn8W5S
See if you can guess which came from an iPhone 12 Pro, and which came from a Nikon D70 (about a $100 camera body at this point).
That all said, the entire point was... expecting anything more than 12MP out of an iPhone is absolutely pointless because the sensor size and optics simply aren't there to actually produce a better photo.
That's just the iPhone's default JPEG processing. You can avoid that by shooting RAW. I posted a comparison in this comment: https://news.ycombinator.com/item?id=27769511
Not sure what is going on with your comparison crop. The phone photo seems to actually be out of focus in the relevant area, not to mention overexposed.
JPEG uses DCT with fixed block sizes. The results of the two file formats are similar.
The main issue, as far as I understand it, is that the tiny sensor and tiny lens at some point just end up limiting the effective resolution. You can capture more pixels, but you never really increase the actual effective resolution of the end product. At that point you're not accomplishing much more than scaling up a lower resolution picture.
The "computational photography" usually ends up just vastly over-doing the sharpening and creating artifacts, and leaving weirdly smooth gaps where there was nothing to sharpen but the camera completely failed to pick up detail.
Just a couple random photos I grabbed from a day I took my kid to the park, but not necessarily like... scientific-paper worthy examples: https://imgur.com/a/7bn8W5S
One's taken with my iPhone 12 Pro, one with a Nikon D70.
Mainly, though, I base this off of trying to print any of the photos I've taken, and maybe someone else can hop in and explain this for me. I could be way off base here.
Printing photos from my iPhone at 4x6 they all look like someone took a really soft or slightly out-of-focus photo, applied way too much sharpening, and printed it. Printing a photo from my D70 generally still looks crisp and natural. Even if I blow it up beyond the actual resolution, it looks pixelated... not soft and then sharpened. This is even though the iPhone has twice the resolution of the D70.
For it's price it is pretty unbeatable, if you're in the MFT ecosystem.
The computational composition (I wouldn't call it photography) done by modern phones also just looks weird. For example, trying to detect foreground and artificially blur the background, leaving a weird in-focus-yet-distant halo of texture around things.
In studio light situations, fstoppers did a video years ago that had the iPhone (6 maybe?) produce pictures that rivaled the dslr at the time.
It's simply more complicated than infinitely better.
If you’re still skeptical, consider a pinhole camera with no lens capturing an image at near infinite focus. Adding a piece of glass that refracts the light can’t perform any better than pure parallel rays of light.
Bigger lenses mean more light. More light means better signal to noise ratio. That's an objective metric of image quality.
Phone cameras can try to compensate for this by taking long exposures - actually videos - and warp the frames into alignment to remove blur, but it's a losing game if there's a lot of motion. Or the camera can try and guess the textures and replace them with similar ones, like the Gigapixel AI and similar services do - but then it's starting to move away from capturing the actual scene.
I mean, most of the photos I take are with my phone, because it's the handiest available camera. I don't think people talking about quality from dedicated cameras are trying to say that they're better than phones as a practical tool for taking everyday photos. But you do lose some quality and computational tricks can't get it all back.
And with a moving subject, a lens obviously makes a sharper picture, since the pinhole requires such a long exposure.
Computational photography can overcome some of the optical limitations of the small sensor. E.g. you need a really good camera to get the dynamic range that is available with iPhone HDR.
Old camera (I am not aware of any modern cameras having 6mpx sensors) is an old camera. Cheap glass is a cheap glass. The bigger the sensor the better your lens has to be optically, because light is bent at the bigger angles.
The two most obvious areas (ergonomics and interchangeable lenses aside) of advantages of the "proper" cameras with bigger sensors are low-light performance and "bokeh". Otoh, if you want to have as much in focus as possible small sensors win.
The angle that the light is bent depends on the angle of view, not the size of the sensor.
And sensor size, that's why I don't understand that metric. Yeah the resolution is right eg. ~8Kx4K
Bigger pixel is the difference apparently
For two sensors of the same resolution (= number of photosensitive cells), a physically larger sensor will have a greater number of photons hitting each photosite. This means better signal-to-noise ratio (fewer photons = base electrical noise is greater in relation to the signal) and dynamic range.
So inescapably, due to basic laws of physics, for two identically implemented sensors of the same resolution, the larger one will always be better.
This doesn't mean you can't make miniscule sensors with 41MP -- you certainly can. There can even be advantages to doing so. The Nokia PureView cameras were based on a novel concept that by capturing very high resolution images (41MP) you can then smooth out the noise (because it is essentially random) while retaining huge amounts of detail if you downsample them to a reasonable size (something like 12MP? I forget what it does exactly) in post-processing. It is a tradeoff -- you trade worse dynamic range for better detail -- but it worked really well for a smartphone at the time.
If you were to pixel peep the 41 MP files before downsampling they would look horrible, especially at higher sensitivities.
Nitpick: this assumes that you're holding the f number constant. In practice smaller sensors tend to be used with smaller f numbers, which somewhat offsets the effect (especially if you are not someone who's given to shooting everything wide open on your DSLR).
A more useful way to think about it is to forget about the sensor and just consider the absolute diameter of the aperture (for a given angle of view). Your phone's aperture is a few mm in diameter. If you're shooting at the same angle of view with your DSLR, then the amount of additional light hitting its sensor (as compared to the phone) is in proportion to the additional diameter of the DSLR's aperture. So if you're shooting at, say, f16, you may not be getting any more light than the phone is at f1.8.
Well most phone cameras seem to be around f/2, some slightly above, some a little below. The archetypal nifty fifty is f/1.8 or f/2 as well, and primes in that range are usually available for most applications and reasonable in price. Slightly slower primes at f/2.8 are often also available and cheaper. So dit-for-dat, you'd expect a full-frame camera to have at least 6 EVs lower noise than your average 1/3.something inch phone camera sensor (crop factor of ~10, area difference of ~100, ld(100) = 6...).
Your entrance pupil metric is really just a roundabout way to compensate for the crop factor of the sensor to get to the same FoV. The relevant property for exposure is the f-stop.
I do think that focusing on sensor size is unhelpful when thinking about noise levels. Big sensors do not magically collect more light simply in virtue of being bigger. They can only do so if you’re able to put a bigger hole in front of them (again, holding constant the angle of view). The use of very wide apertures is inherently more practical with smaller sensors.
As for using wide apertures with a DSLR, this is of course possible, but only in cases where a shallow depth of field is acceptable. On a cell phone camera f1.8 will almost always give sufficient depth of field. Realistically speaking most photos on a DSLR will be taken a few stops down from that.
It’s undoubtedly the case that DSLRs have an advantage over phones in terms of noise levels, but you have to consider the whole optical system to estimate the magnitude of the difference, not just the size of the sensor.
Random side note: it is cool how we adjust our perception of HD eg. back then 480P was "pretty good" ha.
Good lord just looked up one of the Nokia phones and it has 5 cameras on it dang...
It's a slow process moving away from this once you've drilled it into people's heads with marketing. It's just like how MHz used to be an easy and fairly reliable way to compare CPU's in the 90's and it took a really long time for the general public to understand that clock speed isn't all that matters.
I don't know the storage capacity or resolution of my phone, nor the engine size of my car because they're good enough.
That's why you see a focus on showing off the end results now, like Apple's rebooted "Shot on iPhone" campaign or Samsung's current photo competition restricted to photos taken with a Galaxy S21
Most DSLR/mirrorless cameras have a 3:2 ratio, whereas smartphone cameras tend to have a 4:3 ratio.
If you want to make a smartphone camera sensor that can do 3840 pixels across that can shoot 4k, the 4:3 sensor would need to be 2880 pixels tall. So 3840*2880 = 11.1MP would be the bare minimum size for a smartphone camera that can shoot 4K.
In practice you'd want something larger than that, so you can do DCI 4K (which is a bit higher resolution) and do things like gyroscopically assisted electronic image stabilization (which benefits from a few extra pixels around the border). The iPhone can probably implement video stabilization that rivals GoPro's HyperSmooth stabilization.
I guess it really depends on the aspect ratio you're going for.
a) nothing stopping them from faking it, plenty of crappy hardware will record files in an upscaled resolution (see webcams especially for this)
b) you probably don't want a larger version of the output of that device anyways.
I mean even a 130 dollar phone has a 13 megapixel camera. https://www.amazon.com/BLU-Android-Factory-Unlocked-Display/...
If people think it’s interesting (ad, or not), it will be promoted. If not, it won’t live long.